Difference between revisions of "Benchmarks Game"

From HaskellWiki
Jump to navigation Jump to search
(→‎Old Single Core Benchmarks: Updated links (solved redirects))
(→‎Submitting guide: Updated links and GHC version)
(5 intermediate revisions by the same user not shown)
Line 1: Line 1:
This is for entries to [http://shootout.alioth.debian.org/index.php The Computer Language Benchmarks Game] (Formerly known as The Great Language Shootout). You can see [http://shootout.alioth.debian.org/u64q/benchmark.php?test=all&lang=all the current Haskell results].
+
This is for entries to [https://salsa.debian.org/benchmarksgame-team/benchmarksgame The Computer Language Benchmarks Game] (Formerly known as The Great Language Shootout). You can see [https://salsa.debian.org/benchmarksgame-team/benchmarksgame/tree/master/public/data the current results as data files].
  +
  +
The old site was closed at 2018-05-01, the new site is not fully functional yet (at 2018-05-17), see the
  +
[https://www.reddit.com/r/programming/comments/88jg07/the_computer_language_benchmarks_games/ announcement].
 
 
 
[[Image:Chartvs.png|center|frame|Haskell entries compared to C on the quad core, Feb 2009]]
 
[[Image:Chartvs.png|center|frame|Haskell entries compared to C on the quad core, Feb 2009]]
Line 11: Line 14:
 
== Submitting guide ==
 
== Submitting guide ==
   
Now that GHC 6.10 is installed on the Benchmarks Game servers, please make full use of it! There is an immediate need for a complete revamp of all the entries to fix and optimise them for this version of GHC. Please pick a program and contribute!
+
Now that GHC 8.8.1 is installed on the Benchmarks Game servers, please make full use of it! There is an immediate need for a complete revamp of all the entries to fix and optimise them for this version of GHC. Please pick a program and contribute!
  +
  +
GHC change information since version 8.0.1 can be found on the [https://www.haskell.org/ghc/blog.html GHC Developer Blog]
   
 
GHC 6.10 gave us the option of
 
GHC 6.10 gave us the option of
Line 23: Line 28:
   
 
GHC 6.6 gave the option of using:
 
GHC 6.6 gave the option of using:
* [http://haskell.org/ghc/docs/latest/html/libraries/base/Data-ByteString.html Data.ByteString]
+
* [http://www.haskell.org/ghc/docs/latest/html/libraries/base/Data-ByteString.html Data.ByteString]
* [http://haskell.org/ghc/docs/latest/html/libraries/regex-posix/Text-Regex-Posix-ByteString.html Text.Regex.*]
+
* [http://www.haskell.org/ghc/docs/latest/html/libraries/regex-posix/Text-Regex-Posix-ByteString.html Text.Regex.*]
* [http://www.haskell.org/ghc/docs/latest/html/users_guide/sec-bang-patterns.html bang patterns]
+
* [http://www.haskell.org/ghc/docs/latest/html/users_guide/bang-patterns.html bang patterns]
 
which should greatly improve clarity and brevity. Line count is not
 
which should greatly improve clarity and brevity. Line count is not
 
significant now (code is gzipped to determine size), so aim for maximum
 
significant now (code is gzipped to determine size), so aim for maximum
Line 46: Line 51:
 
<haskell>
 
<haskell>
 
--
 
--
-- The Computer Language Benchmarks Game, http://benchmarksgame.alioth.debian.org/
+
-- The Computer Language Benchmarks Game, https://benchmarksgame-team.pages.debian.net/benchmarksgame/
-- Haskell Benchmarks Game entries, http://www.haskell.org/haskellwiki/Benchmarks_Game
+
-- Haskell Benchmarks Game entries, https://wiki.haskell.org/Benchmarks_Game
 
--
 
--
 
-- Simon Marlow
 
-- Simon Marlow
Line 56: Line 61:
 
== Results ==
 
== Results ==
   
  +
Results for all benchmarks: [https://salsa.debian.org/benchmarksgame-team/benchmarksgame/tree/master/public/data]
Haskell results for all benchmarks: [http://shootout.alioth.debian.org/gp4/benchmark.php?test=all&lang=ghc&lang2=ghc Gentoo/P4] and [http://shootout.alioth.debian.org/benchmark.php?test=all&lang=ghc&lang2=ghc Debian/AMD Sempron]
 
   
 
== Todo ==
== Old Single Core Benchmarks ==
 
   
 
The current benchmarks can highlight weak spots in libraries and
* [[Shootout/Binary trees | Binary trees]] - for [http://benchmarksgame.alioth.debian.org/u32/benchmark.php?test=binarytrees&lang=all Gentoo/P4], [http://benchmarksgame.alioth.debian.org/u32/performance.php?test=binarytrees Debian/AMD Sempron]
 
 
compilers that need improving. Some things noticed included:
   
 
* hashtable performance is poor. A simple binding to a basic C hashtable would be very useful
* [[Shootout/Cheap concurrency | Cheap concurrency]] (*)
 
 
* it would be nice to have tre regexes in the core libs (instead of POSIX ones)
   
 
''Why tre regexes? The libtre code is buggy, though if the shootout avoids the bugs then I guess one might want to use it. Would libpcre be a better choice?''--[[User:ChrisKuklewicz|ChrisKuklewicz]] 00:21, 24 February 2009 (UTC)
* [[Shootout/Thread ring | Thread-ring]] - for [http://benchmarksgame.alioth.debian.org/u32/benchmark.php?test=thread-ring&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=thread-ring&lang=all Debian/AMD Sempron]
 
   
* [[Shootout/Chameneos | Chameneos]] (*)
 
   
 
== Obsolete benchmarks ==
* [[Shootout/Fannkuch | Fannkuch-redux]] - for [http://benchmarksgame.alioth.debian.org/u32/benchmark.php?test=fannkuch&lang=all Gentoo/P4], [http://benchmarksgame.alioth.debian.org/u32/performance.php?test=fannkuchredux Debian/AMD Sempron]
 
   
 
* [[Shootout/Takfp | Takfp]]
* [[Shootout/Fasta | Fasta]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=fasta&lang=all Gentoo/P4], [http://benchmarksgame.alioth.debian.org/u32/performance.php?test=fasta Debian/AMD Sempron]
 
   
 
* [[Shootout/Ack | Ack]]
* [[Shootout/Knucleotide | K-nucleotide]] - for [http://benchmarksgame.alioth.debian.org/u32/benchmark.php?test=knucleotide&lang=all Gentoo/P4], [http://benchmarksgame.alioth.debian.org/u32/performance.php?test=knucleotide Debian/AMD Sempron]
 
   
  +
* [[Shootout/Harmonic | Harmonic]]
* [[Shootout/Mandelbrot | Mandelbrot]] - for [http://benchmarksgame.alioth.debian.org/u32/benchmark.php?test=mandelbrot&lang=all Gentoo/P4] , [http://benchmarksgame.alioth.debian.org/u32/performance.php?test=mandelbrot Sempron]
 
   
  +
* [[Shootout/Healthcare | Healthcare]]
* [[Shootout/Nbody | N-body]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=nbody&lang=all Gentoo/P4] , [http://benchmarksgame.alioth.debian.org/u32/performance.php?test=nbody Debian/AMD Sempron]
 
   
* [[Shootout/Nsieve | N-sieve]] (*)
+
* [[Shootout/Nsieve | N-sieve]]
   
* [[Shootout/Nsieve Bits | N-sieve Bits]] (*)
+
* [[Shootout/Nsieve Bits | N-sieve Bits]]
   
* [[Shootout/Partial sums | Partial sums]] (*)
+
* [[Shootout/Partial sums | Partial sums]]
   
 
* [[Shootout/Cheap concurrency | Cheap concurrency]]
* [[Shootout/Pidigits | Pidigits]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=pidigits&lang=all Gentoo/P4] , [http://benchmarksgame.alioth.debian.org/u32/performance.php?test=pidigits Debian/AMD Sempron]
 
 
* [[Shootout/Random | Random]] (*)
 
 
* [[Shootout/Recursive | Recursive]] (*)
 
 
* [[Shootout/Regex DNA | Regex DNA]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=regexdna&lang=all Gentoo/P4], [http://benchmarksgame.alioth.debian.org/u32/performance.php?test=regexdna Debian/AMD Sempron]
 
 
* [[Shootout/Reverse complement | Reverse complement]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=revcomp&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=revcomp&lang=all Debian/AMD Sempron]
 
 
* [[Shootout/Spectral | Spectral]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=spectralnorm&lang=all Gentoo/P4], [http://benchmarksgame.alioth.debian.org/u32/performance.php?test=spectralnorm Debian/AMD Sempron]
 
 
* [[Shootout/SumFile | SumFile]] (*)
 
 
(*) Not on the Benchmarks Game site anymore
 
 
== Todo ==
 
 
The current benchmarks can highlight weak spots in libraries and
 
compilers that need improving. Some things noticed included:
 
 
* hashtable performance is poor. A simple binding to a basic C hashtable would be very useful
 
* it would be nice to have tre regexes in the core libs (instead of POSIX ones)
 
 
''Why tre regexes? The libtre code is buggy, though if the shootout avoids the bugs then I guess one might want to use it. Would libpcre be a better choice?''--[[User:ChrisKuklewicz|ChrisKuklewicz]] 00:21, 24 February 2009 (UTC)
 
 
== Obsolete benchmarks ==
 
   
 
* [[Shootout/Chameneos | Chameneos]]
* [[Shootout/Takfp]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=takfp&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=takfp&lang=all Debian/AMD Sempron]
 
   
  +
* [[Shootout/Random | Random]]
* [[Shootout/Ack]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=ackermann&lang=all Gentoo/P4], [http://shootout.alioth.debian.org/benchmark.php?test=ackermann&lang=all Debian/AMD Sempron]
 
   
 
* [[Shootout/Recursive | Recursive]]
* [[Shootout/Harmonic]] - for [http://shootout.alioth.debian.org/gp4/benchmark.php?test=harmonic&lang=all Gentoo/P4] , [http://shootout.alioth.debian.org/benchmark.php?test=harmonic&lang=all Debian/AMD Sempron]
 
   
  +
* [[Shootout/SumFile | SumFile]]
== Future benchmarks ==
 
   
* [[Shootout/Healthcare]] - for [http://shootout.alioth.debian.org/sandbox/benchmark.php?test=health&lang=all Debian/AMD Sempron]
 
   
 
[[Category:Contests]]
 
[[Category:Contests]]

Revision as of 23:13, 18 April 2020

This is for entries to The Computer Language Benchmarks Game (Formerly known as The Great Language Shootout). You can see the current results as data files.

The old site was closed at 2018-05-01, the new site is not fully functional yet (at 2018-05-17), see the announcement.

Haskell entries compared to C on the quad core, Feb 2009

Parallel implementations

New implementations written for parallelism. You should almost certainly be working on these only:

Submitting guide

Now that GHC 8.8.1 is installed on the Benchmarks Game servers, please make full use of it! There is an immediate need for a complete revamp of all the entries to fix and optimise them for this version of GHC. Please pick a program and contribute!

GHC change information since version 8.0.1 can be found on the GHC Developer Blog

GHC 6.10 gave us the option of

  • parallel garbage collector

GHC 6.8 gave us the option of

  • `par`
  • SMP runtime
  • new regex libs
  • lots of other stuff

GHC 6.6 gave the option of using:

which should greatly improve clarity and brevity. Line count is not significant now (code is gzipped to determine size), so aim for maximum clarity.

Suggested best practices for submitting an entry to the Benchmarks Game include:

  • Posting the proposed code to the community to allow for a best of breed entry. Proposed code is here on the wiki and often in the Haskell-cafe mailing list. This has already helped improve the submitted code.
  • Provide complete performance measurements against the current best entry.
  • Creating clean, functional-style code or fast, imperative-style code. There are multiple versions ("GHC", "GHC#2", "GHC#3", "GHC#4") of GHC codes on many of the benchmarks with no clear rational for the different versions. It'd be better to have entries for "GHC" be clean, short, functional code and entries for "GHC-#2" be fastest, probably-ugliest code.
  • When you submit via the trac system, record in the wiki section for that entry a link to the submission.
  • Check for gzip file size before you submit -- lots of points to be had there.
  • Try different compile options and the other ideas on the Performance/GHC page.
--
-- The Computer Language Benchmarks Game, https://benchmarksgame-team.pages.debian.net/benchmarksgame/
-- Haskell Benchmarks Game entries, https://wiki.haskell.org/Benchmarks_Game
-- 
-- Simon Marlow
-- Shortened by Don Stewart
--

Results

Results for all benchmarks: [1]

Todo

The current benchmarks can highlight weak spots in libraries and compilers that need improving. Some things noticed included:

  • hashtable performance is poor. A simple binding to a basic C hashtable would be very useful
  • it would be nice to have tre regexes in the core libs (instead of POSIX ones)

Why tre regexes? The libtre code is buggy, though if the shootout avoids the bugs then I guess one might want to use it. Would libpcre be a better choice?--ChrisKuklewicz 00:21, 24 February 2009 (UTC)


Obsolete benchmarks