Software Engineering Stack Exchange
Software Engineering Stack Exchange

Questions tagged [benchmarking]

42 questions
29
votes
5 answers

Benchmarking, why discard lowest time?

I've quite frequently seen benchmarks where the tester discarded the highest and the lowest time out of N runs. Discarding the highest time I understand; it's probably high because of some other processes running suddenly demanding more CPU. But…
pepoluan
  • 499
  • 4
  • 10
23
votes
3 answers

Does anyone do hardware benchmarks on compiling code?

I've seen a bunch of sites that benchmark new hardware on gaming performance, zipping some files, encoding a movie, or whatever. Are there any that test the impact of new hardware (like SSDs, new CPUs, RAM speeds, or whatever) on compile and link…
Colen
  • 536
19
votes
3 answers

Fastest Functional Language

I've recently been delving into functional programming especially Haskell and F#, the prior more so. After some googling around I could not find a benchmark comparison of the more prominent functional languages (Scala,F# etc). I know it's not…
Farouk
  • 309
14
votes
4 answers

Benchmarking databases

I see lots of discussions flying around about performance of db 'x' or that moving from 'x' to 'y' improved our site performance. I'm yet to see proper benchmarking that works across different types of databases. Is it possible to write a…
Dan McGrath
  • 11,181
  • 6
  • 57
  • 82
10
votes
7 answers

Does memoization skew benchmarks?

I'm writing a program that works with directed acyclic graphs and performs rewriting operations on them. Sometimes the graphs are small, sometimes they can be very large and grow as the program advances. In the context of my program, it is common…
Foxy
  • 621
8
votes
5 answers

Does profiling without benchmarking lead to micro-optimization?

Read first: For a definition of the two terms ("profiling" and "benchmarking") and the need for distinguishing them, please read this answer to a previous question. I have to admit that until I saw Winston Ewert's answer, I have never thought of the…
rwong
  • 17,140
7
votes
2 answers

Where is the evidence for the performance of Node.js?

A lot of developers, especially ones who haven't used Node.js in production, seem to believe that Node.js is faster than other interpreted languages such as PHP, Python, and Ruby. These claims include: Node.js/V8 is x times faster than y…
Ten Bitcomb
  • 1,174
6
votes
11 answers

If python compiles to assembly and an OS is written in it, will it compete favorably with C in benchmarks?

Ok, I have used the word python in the question, but it well could be language agnostic in that: If a language X has a well optimised compiler targeting assembly and an OS is written in that language, then will it compete favorably with C in…
check123
  • 1,327
5
votes
5 answers

How to test for performance regression?

I am working on a refactor on a certain package (I can give details if asked). The package involves a clever lazy evaluation of a sort of nested sequence of arithmetic operations. If the numerical algorithm is implemented in the naive way you will…
Jagerber48
  • 343
5
votes
2 answers

Estimate if benchmark sampling difference is significative

In order to estimate the fastest function in various cases in a non fully deterministic context, I'm running an experiment calling one or the other at random and recording the duration of the operation on the production server. I get this kind of…
Denys Séguret
  • 1,454
  • 1
  • 10
  • 14
5
votes
3 answers

Is running "milli"-benchmarks a good idea?

I just came across the Caliper project and it looks very nice. Reading the introduction to microbenchmarks, one gets the feeling that the developers would not suggest to use the framework if the benchmark takes longer than a second or so. I looked…
Konstantin Weitz
  • 153
  • 4
5
votes
1 answer

Use of a profiler tool to aid in the analysis of a brute force algorithm in Java

I was asked to profile (using some tool, such as YourKit or JVisualVM) a couple of implementations of the Traveling Salesman Problem (find the minimum path that visits all the given set of cities), in the context of a Algorithms Analysis and Design…
devoured elysium
  • 592
4
votes
4 answers

How to manage code experiments meant to stay permanently with git?

I have an asm/C code which implements some image filters. The objective of this project is experimentation with different implementations, benchmarking, plotting and reporting data to write a paper with relevant insights. So for example one…
Damian Pereira
  • 143
4
votes
2 answers

How to test the render speed of my solution in a web browser?

Ok, I need to test the speed of my solution in a web browser, but I have some problems, there are 2 versions of the web solution, the original one that is on server A and the "fixed" version that is on server B. I have VS2010 Ultimate, so I can make…
Luis
  • 535
4
votes
2 answers

Benchmarking CPU processing power

Provided that many tools for computers benchmarking are available already, I'd like to write my own, starting with processing power measurement. I'd like to write it in C under Linux, but other language alternatives are welcome. I thought starting…
Federico Zancan
  • 1,463
1
2 3