38 Performance

38.1 Microbenchmarking

  1. Q: Instead of using microbenchmark(), you could use the built-in function system.time(). But system.time() is much less precise, so you’ll need to repeat each operation many times with a loop, and then divide to find the average time of each operation, as in the code below.

    How do the estimates from system.time() compare to those from microbenchmark()? Why are they different?

  2. Q: Here are two other ways to compute the square root of a vector. Which do you think will be fastest? Which will be slowest? Use microbenchmarking to test your answers.

    A: The second one looks more complex, but you never know…unless you test it.

  3. Q: Use microbenchmarking to rank the basic arithmetic operators (+, -, *, /, and ^) in terms of their speed. Visualise the results. Compare the speed of arithmetic on integers vs. doubles.

    A: Since I am on a Windows system, where these short execution times are hard to measure, I just ran the following code on a linux and paste the results here:

    To visualise and compare the results, we make some short spaghetties:

  4. Q: You can change the units in which the microbenchmark results are expressed with the unit parameter. Use unit = "eps" to show the number of evaluations needed to take 1 second. Repeat the benchmarks above with the eps unit. How does this change your intuition for performance?

38.2 Language performance

  1. Q: scan() has the most arguments (21) of any base function. About how much time does it take to make 21 promises each time scan is called? Given a simple input (e.g., scan(text = "1 2 3", quiet = T)) what proportion of the total run time is due to creating those promises?

    A: According to the textbook every extra argument slows the function down by approximately 20 nanoseconds, which I can’t reproduce on my system:

    However, for now we just assume that 20 nanoseconds are correct and in kind of doubt, we recommend to benchmark this value individually. With this assumption we calculate 21 * 20 = 420 nanoseconds of extra time for each call of scan().

    For a percentage, we first benchmark a simple call of scan():

    This lets us calculate, that ~1.28% of the median run time are caused by the extra arguments.

  2. Q: Read “Evaluating the Design of the R Language”. What other aspects of the R-language slow it down? Construct microbenchmarks to illustrate.

  3. Q: How does the performance of S3 method dispatch change with the length of the class vector? How does performance of S4 method dispatch change with number of superclasses? How about RC?

  4. Q: What is the cost of multiple inheritance and multiple dispatch on S4 method dispatch?

  5. Q: Why is the cost of name lookup less for functions in the base package?

38.3 Implementations performance

  1. Q: The performance characteristics of squish_ife(), squish_p(), and squish_in_place() vary considerably with the size of x. Explore the differences. Which sizes lead to the biggest and smallest differences?

  2. Q: Compare the performance costs of extracting an element from a list, a column from a matrix, and a column from a data frame. Do the same for rows.