Difference between revisions of "Benchmarks Game"
(→Submitting guide: clarify...) |
|||
(15 intermediate revisions by 4 users not shown) | |||
Line 1: | Line 1: | ||
− | This is for entries to [http://shootout.alioth.debian.org/index.php The Great Language Shootout]. |
+ | This is for entries to [http://shootout.alioth.debian.org/index.php The Computer Language Benchmarks Game] (Formerly known as The Great Language Shootout). You can see [http://shootout.alioth.debian.org/u64q/benchmark.php?test=all&lang=all the current Haskell results]. |
+ | |||
+ | [[Image:Chartvs.png|center|frame|Haskell entries compared to C on the quad core, Feb 2009]] |
||
+ | |||
+ | == Parallel implementations == |
||
+ | New implementations written for parallelism. You should almost certainly |
||
+ | be working on these only: |
||
+ | |||
+ | * [[/Parallel]] |
||
== Submitting guide == |
== Submitting guide == |
||
− | Now that GHC 6. |
+ | Now that GHC 6.10 is installed on the Benchmarks Game servers, please make full use of it! There is an immediate need for a complete revamp of all the entries to fix and optimise them for this version of GHC. Please pick a program and contribute! |
+ | |||
+ | GHC 6.10 gave us the option of |
||
+ | * parallel garbage collector |
||
+ | |||
+ | GHC 6.8 gave us the option of |
||
+ | * `par` |
||
+ | * SMP runtime |
||
+ | * new regex libs |
||
+ | * lots of other stuff |
||
+ | |||
+ | GHC 6.6 gave the option of using: |
||
* [http://haskell.org/ghc/docs/latest/html/libraries/base/Data-ByteString.html Data.ByteString] |
* [http://haskell.org/ghc/docs/latest/html/libraries/base/Data-ByteString.html Data.ByteString] |
||
* [http://haskell.org/ghc/docs/latest/html/libraries/regex-posix/Text-Regex-Posix-ByteString.html Text.Regex.*] |
* [http://haskell.org/ghc/docs/latest/html/libraries/regex-posix/Text-Regex-Posix-ByteString.html Text.Regex.*] |
||
Line 11: | Line 30: | ||
clarity. |
clarity. |
||
− | Suggested best practices for |
+ | Suggested best practices for submitting an entry to the Benchmarks Game include: |
− | * Posting the proposed code to the community to allow for a best of breed entry. Proposed code is here on the wiki and often in the |
+ | * Posting the proposed code to the community to allow for a best of breed entry. Proposed code is here on the wiki and often in the Haskell-cafe mailing list. This has already helped improve the submitted code. |
* Provide complete performance measurements against the current best entry. |
* Provide complete performance measurements against the current best entry. |
||
Line 22: | Line 41: | ||
* Check for gzip file size before you submit -- lots of points to be had there. |
* Check for gzip file size before you submit -- lots of points to be had there. |
||
+ | |||
+ | * Try different compile options and the other ideas on the [[Performance/GHC]] page. |
||
<haskell> |
<haskell> |
||
Line 37: | Line 58: | ||
Haskell results for all benchmarks: [http://shootout.alioth.debian.org/gp4/benchmark.php?test=all&lang=ghc&lang2=ghc Gentoo/P4] and [http://shootout.alioth.debian.org/benchmark.php?test=all&lang=ghc&lang2=ghc Debian/AMD Sempron] |
Haskell results for all benchmarks: [http://shootout.alioth.debian.org/gp4/benchmark.php?test=all&lang=ghc&lang2=ghc Gentoo/P4] and [http://shootout.alioth.debian.org/benchmark.php?test=all&lang=ghc&lang2=ghc Debian/AMD Sempron] |
||
+ | == Old Single Core Benchmarks == |
||
− | == Current benchmarks == |
||
− | * [[/Binary trees]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=binarytrees&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=binarytrees&lang=all Debian/AMD Sempron] |
+ | * [[Shootout/Binary trees]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=binarytrees&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=binarytrees&lang=all Debian/AMD Sempron] |
− | * [[/Cheap concurrency]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=thread-x&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=thread-x&lang=all Debian/AMD Sempron] |
+ | * [[Shootout/Cheap concurrency]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=thread-x&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=thread-x&lang=all Debian/AMD Sempron] |
− | * [[/Thread ring]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=thread-ring&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=thread-ring&lang=all Debian/AMD Sempron] |
+ | * [[Shootout/Thread ring]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=thread-ring&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=thread-ring&lang=all Debian/AMD Sempron] |
− | * [[/Chameneos]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=chameneos&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=chameneos&lang=all Debian/AMD Sempron] |
+ | * [[Shootout/Chameneos]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=chameneos&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=chameneos&lang=all Debian/AMD Sempron] |
− | * [[/Fannkuch]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=fannkuch&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=fannkuch&lang=all Debian/AMD Sempron] |
+ | * [[Shootout/Fannkuch]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=fannkuch&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=fannkuch&lang=all Debian/AMD Sempron] |
− | * [[/Fasta]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=fasta&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=fasta&lang=all Debian/AMD Sempron] |
+ | * [[Shootout/Fasta]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=fasta&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=fasta&lang=all Debian/AMD Sempron] |
− | * [[/Knucleotide]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=knucleotide&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=knucleotide&lang=all Debian/AMD Sempron] |
+ | * [[Shootout/Knucleotide]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=knucleotide&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=knucleotide&lang=all Debian/AMD Sempron] |
− | * [[/Mandelbrot]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=mandelbrot&lang=all Gentoo/P4] , [http://shootout.alioth.debian.org/benchmark.php?test=mandelbrot&lang=all Debian/AMD Sempron] |
+ | * [[Shootout/Mandelbrot]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=mandelbrot&lang=all Gentoo/P4] , [http://shootout.alioth.debian.org/benchmark.php?test=mandelbrot&lang=all Debian/AMD Sempron] |
− | * [[/Nbody]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=nbody&lang=all Gentoo/P4] , [http://shootout.alioth.debian.org/benchmark.php?test=nbody&lang=all Debian/AMD Sempron] |
+ | * [[Shootout/Nbody]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=nbody&lang=all Gentoo/P4] , [http://shootout.alioth.debian.org/benchmark.php?test=nbody&lang=all Debian/AMD Sempron] |
− | * [[/Nsieve]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=nsieve&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=nsieve&lang=all Debian/AMD Sempron] |
+ | * [[Shootout/Nsieve]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=nsieve&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=nsieve&lang=all Debian/AMD Sempron] |
− | * [[/Nsieve Bits]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=nsievebits&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=nsievebits&lang=all Debian/AMD Sempron] |
+ | * [[Shootout/Nsieve Bits]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=nsievebits&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=nsievebits&lang=all Debian/AMD Sempron] |
− | * [[/Partial sums]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=partialsums&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=partialsums&lang=all Debian/AMD Sempron] |
+ | * [[Shootout/Partial sums]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=partialsums&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=partialsums&lang=all Debian/AMD Sempron] |
− | * [[/Pidigits]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=pidigits&lang=all Gentoo/P4] , [http://shootout.alioth.debian.org/benchmark.php?test=pidigits&lang=all Debian/AMD Sempron] |
+ | * [[Shootout/Pidigits]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=pidigits&lang=all Gentoo/P4] , [http://shootout.alioth.debian.org/benchmark.php?test=pidigits&lang=all Debian/AMD Sempron] |
− | * [[/Random]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=random&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=random&lang=all Debian/AMD Sempron] |
+ | * [[Shootout/Random]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=random&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=random&lang=all Debian/AMD Sempron] |
− | * [[/Recursive]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=recursive&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=recursive&lang=all Debian/AMD Sempron] |
+ | * [[Shootout/Recursive]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=recursive&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=recursive&lang=all Debian/AMD Sempron] |
− | * [[/Regex DNA]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=regexdna&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=regexdna&lang=all Debian/AMD Sempron] |
+ | * [[Shootout/Regex DNA]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=regexdna&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=regexdna&lang=all Debian/AMD Sempron] |
− | * [[/Reverse complement]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=revcomp&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=revcomp&lang=all Debian/AMD Sempron] |
+ | * [[Shootout/Reverse complement]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=revcomp&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=revcomp&lang=all Debian/AMD Sempron] |
− | * [[/Spectral]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=spectralnorm&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=spectralnorm&lang=all Debian/AMD Sempron] |
+ | * [[Shootout/Spectral]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=spectralnorm&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=spectralnorm&lang=all Debian/AMD Sempron] |
− | * [[/SumFile]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=sumcol&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=sumcol&lang=all Debian/AMD Sempron] |
+ | * [[Shootout/SumFile]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=sumcol&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=sumcol&lang=all Debian/AMD Sempron] |
== Todo == |
== Todo == |
||
Line 82: | Line 103: | ||
compilers that need improving. Some things noticed included: |
compilers that need improving. Some things noticed included: |
||
− | * hashtable |
+ | * hashtable performance is poor. A simple binding to a basic C hashtable would be very useful |
− | * it would be nice to have tre regexes in the core libs (instead of |
+ | * it would be nice to have tre regexes in the core libs (instead of POSIX ones) |
+ | |||
+ | ''Why tre regexes? The libtre code is buggy, though if the shootout avoids the bugs then I guess one might want to use it. Would libpcre be a better choice?''--[[User:ChrisKuklewicz|ChrisKuklewicz]] 00:21, 24 February 2009 (UTC) |
||
== Obsolete benchmarks == |
== Obsolete benchmarks == |
||
− | * [[/Takfp]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=takfp&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=takfp&lang=all Debian/AMD Sempron] |
+ | * [[Shootout/Takfp]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=takfp&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=takfp&lang=all Debian/AMD Sempron] |
− | * [[/Ack]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=ackermann&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=ackermann&lang=all Debian/AMD Sempron] |
+ | * [[Shootout/Ack]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=ackermann&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=ackermann&lang=all Debian/AMD Sempron] |
− | * [[/Harmonic]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=harmonic&lang=all Gentoo/P4] , [http://shootout.alioth.debian.org/benchmark.php?test=harmonic&lang=all Debian/AMD Sempron] |
+ | * [[Shootout/Harmonic]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=harmonic&lang=all Gentoo/P4] , [http://shootout.alioth.debian.org/benchmark.php?test=harmonic&lang=all Debian/AMD Sempron] |
== Future benchmarks == |
== Future benchmarks == |
||
− | * [[/Healthcare]] - for [http://shootout.alioth.debian.org/sandbox/benchmark.php?test=health&lang=all Debian/AMD Sempron] |
+ | * [[Shootout/Healthcare]] - for [http://shootout.alioth.debian.org/sandbox/benchmark.php?test=health&lang=all Debian/AMD Sempron] |
[[Category:Contests]] |
[[Category:Contests]] |
Revision as of 22:47, 22 January 2012
This is for entries to The Computer Language Benchmarks Game (Formerly known as The Great Language Shootout). You can see the current Haskell results.
Parallel implementations
New implementations written for parallelism. You should almost certainly be working on these only:
Submitting guide
Now that GHC 6.10 is installed on the Benchmarks Game servers, please make full use of it! There is an immediate need for a complete revamp of all the entries to fix and optimise them for this version of GHC. Please pick a program and contribute!
GHC 6.10 gave us the option of
- parallel garbage collector
GHC 6.8 gave us the option of
- `par`
- SMP runtime
- new regex libs
- lots of other stuff
GHC 6.6 gave the option of using:
which should greatly improve clarity and brevity. Line count is not significant now (code is gzipped to determine size), so aim for maximum clarity.
Suggested best practices for submitting an entry to the Benchmarks Game include:
- Posting the proposed code to the community to allow for a best of breed entry. Proposed code is here on the wiki and often in the Haskell-cafe mailing list. This has already helped improve the submitted code.
- Provide complete performance measurements against the current best entry.
- Creating clean, functional-style code or fast, imperative-style code. There are multiple versions ("GHC", "GHC#2", "GHC#3", "GHC#4") of GHC codes on many of the benchmarks with no clear rational for the different versions. It'd be better to have entries for "GHC" be clean, short, functional code and entries for "GHC-#2" be fastest, probably-ugliest code.
- When you submit via the trac system, record in the wiki section for that entry a link to the submission.
- Check for gzip file size before you submit -- lots of points to be had there.
- Try different compile options and the other ideas on the Performance/GHC page.
--
-- The Great Computer Language Shootout, http://shootout.alioth.debian.org/
-- Haskell Shootout entries, http://haskell.org/haskellwiki/Great_language_shootout
--
-- Simon Marlow
-- Shortened by Don Stewart
--
Results
Haskell results for all benchmarks: Gentoo/P4 and Debian/AMD Sempron
Old Single Core Benchmarks
Todo
The current benchmarks can highlight weak spots in libraries and compilers that need improving. Some things noticed included:
- hashtable performance is poor. A simple binding to a basic C hashtable would be very useful
- it would be nice to have tre regexes in the core libs (instead of POSIX ones)
Why tre regexes? The libtre code is buggy, though if the shootout avoids the bugs then I guess one might want to use it. Would libpcre be a better choice?--ChrisKuklewicz 00:21, 24 February 2009 (UTC)