Page tree

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

This section includes the performance test results gathered by the SoftInstigate’s development team and includes all the information needed to autonomously reproduce the tests.

Results

Test case 1

Measure the execution time to create 1 million documents with random data, using 200 test threads.

...

 Execution TimeTPS
RESTHeart250s3990 tps
Direct244s4086 tps

Test case 2

Measure the execution time to query a collection 100.000 times, getting 5 documents each time (limit 5) and skipping just 25 documents, under different concurrency levels. 

...

Threads50100200250400500
RESTHeart78s82s78s76s76s76s
Direct97s95s96s96s109s112s

Test case 3

Measure the execution time to query a collection 2.000 times, getting 5 documents each time (limit 5) and skipping just 250.000 documents, under different concurrency levels.

...

Threads124581020405080100200400500
RESTHeart16,28s6,22s5,05s2,53s3,76s3,6s2,98s5,65s9,04s10,74s6,76s9,24s6,76s12,71s
Direct1091s627s324s328s329s325s324s321s321s304s302s305s327s327s

Test case 4

Measure the execution time to query a collection 500 times, getting 5 documents each time (limit 5) skipping more and more documents each time, with a concurrency level of 4.

...

Kskips2505007501.0001.2501.5001.7502.0002.250
RESTHeart2,6s1,91s1,9s1,88s1,58s2,05s1,51s1,52s1,51s
Direct79s156s242s317s375s453s558s601s713s

How we tested

Hardware

MongoDB and RESTHeart running on Sunfire X2200 M2 with 2 CPU with 16 Gbyte of RAM. See full specification here. This is an old server with 2 dual core 2,2GHz AMD Opteron CPUs.

...

The client and the server on the same local network linked by a 10/100 Ethernet switch.

Software

  • Server OS: Ubuntu server 64bit 14.04.1 LTS
  • Client OS: Mac OS X Yosemite 10.10.1
  • MongoDB: 2.6.7
  • RESTHeart: commit 01d403a5db8b765ad5b0a8eec1fda420c392ab58

Configurations

MongoDB: run (without authentication enabled) with the following command

...

  • logging to file off
  • eager-cursor-allocation-linear-slice-heights set to [50]
  • io-threads: 8 
  • worker-threads: 64 

Test cases

We used the brilliant load test tool.

...