Benchmark Performance Testing (aka Individual Time Trials)

[nbolamav]

IBM Cloud Bare Metal

RAM 2x 16GB Micron 16GB DDR4 2Rx4
Processor 2x 2.1GHz Intel Xeon-Skylake (4110-SILVER)
Power Supply 2x SuperMicro PWS-1K02A-1R
Network Card SuperMicro AOC-2UR66-i4XTF
Motherboard SuperMicro X11DPU+_R1.10
GPU NVIDIA TESLA K80

[2020-08-16T02:40:11.985094846Z INFO solana_bench_tps::bench] Node address | Max TPS | Total Transactions [2020-08-16T02:40:11.985104288Z INFO solana_bench_tps::bench] ---------------------+---------------+-------------------- [2020-08-16T02:40:11.985112954Z INFO solana_bench_tps::bench] 127.0.0.1:8003 | 69403.67 | 3158768 [2020-08-16T02:40:11.985125004Z INFO solana_bench_tps::bench] Average max TPS: 69403.67, 0 nodes had 0 TPS [2020-08-16T02:40:11.985134635Z INFO solana_bench_tps::bench] Highest TPS: 69403.67 sampling period 1s max transactions: 3158768 clients: 1 drop rate: 0.62 [2020-08-16T02:40:11.985145821Z INFO solana_bench_tps::bench] Average TPS: 34653.04

[G1715]

why 90? I didn’t see that in the instructions, is that just a number you made up?

[Gnossienli]

Ryzen 3700X,16GB RAM, 1TB SSD Nvme, 1x NVIDIA 2080TI

Only CPU:

Average max TPS: 43961.43, 0 nodes had 0 TPS
Highest TPS: 43961.43 sampling period 1s max transactions: 3392569 clients: 1 drop rate: 0.76
Average TPS: 37507.06

With GPU

Average max TPS: 114793.67, 0 nodes had 0 TPS
Highest TPS: 114793.67 sampling period 1s max transactions: 7336420 clients: 1 drop rate: 0.57
Average TPS: 80683.016

[mbroeken]

VMware running on 1x Intel® Core™ i7-4770 CPU @ 3.40GHz
Virtual machine has 24GB of RAM. 8 cores.
VM is running on Raid 5 10GB NFS Shared Storage using 3x Intel SSD S3700
NVIDIA T4 GPU Enabled
Local Network

bench-tps output:

Sampler 76259.93 TPS, Transactions: 76305, Total transactions: 9468865 over 301 s
Max TPS | Total Transactions
76259.93 | 9468865

Average TPS: 31374.154

[pkrasam]

completed & submitted results via form

[kris]

2x Intel Xeon E5-2620v4 @ 2.1Ghz (8 core/16 HT, 20MB cache each CPU)
64 GB RAM (8x 8GB DD4 2133)
4x 300GB 10K SAS HDD in RAID 5
OS & Account drive separate LVM LVs in same PV
No GPUs
TdS

With default trace level logging on validator
[2020-08-17T18:55:16.858430309Z INFO solana_bench_tps::bench] Average max TPS: 95511.38, 0 nodes had 0 TPS
[2020-08-17T18:55:16.858445036Z INFO solana_bench_tps::bench] Highest TPS: 95511.38 sampling period 1s max transactions: 4197121 clients: 1 drop rate: 0.60
[2020-08-17T18:55:16.858465790Z INFO solana_bench_tps::bench] Average TPS: 46093.523

With RUST_LOG=warn on validator
[2020-08-17T19:12:00.660520328Z INFO solana_bench_tps::bench] Average max TPS: 103514.71, 0 nodes had 0 TPS
[2020-08-17T19:12:00.660528112Z INFO solana_bench_tps::bench] Highest TPS: 103514.71 sampling period 1s max transactions: 4583238 clients: 1 drop rate: 0.55
[2020-08-17T19:12:00.660540872Z INFO solana_bench_tps::bench] Average TPS: 50689.96

[cjremus]

@eric what’s the timeline for this? I have some server reshuffling to do and would like to participate.

[eric]

End of week should be fine!

[G1715]

3960X + 64G + 1x2080Ti
Highest TPS: 185089.05
Average TPS: 155914.94

[ansoncyk]

Submitted via google form just now.

Anson @ Forbole

[ansoncyk]

Did any of you experience issues enabling CUBA for the benchmarking script? There were some hiccup for me using CUDA-11

[G1715]

Hi mate, I believe it wants Cuda 10.1 update 1

As you mentioned, the most recent one is 11, but you can find the older version on nvidia’s website.

[Codonyat]

I think CUDA 11 is not supported by some libraries that are used in Solana.

[ansoncyk]

Thanks G1715 and Godonyat, can you share any reference on this topic if you have it handy? I was able to get it running at the end, but 4 x 2080 Ti display cards only can get my average TPS from 38K to 44K… I am quite disappointed about that.

[Codonyat]

That is indeed a very disappointing improvement with 4x2080ti’s! The bottleneck is obviously somewhere else… How are you on RAM?

For me while I can enable CUDA in my validator, when compiling from source for the benchmark test, it cannot find the libcuda-crypt.so… an so I can only do the no cuda test. Did you do anything in particular to make it work?

[Codonyat]

Check
https://discordapp.com/channels/428295358100013066/560174212967432193/738837077914812470

[melea]

 Average max TPS: 228k

https://www.amd.com/es/products/cpu/amd-epyc-7502p
NVme SSD

Ok, seems like this setup is better

NOTE: No, GPU

[G1715]

I used 10.1 update 1 because that is what the instructions said – you can see it under GPU requirements section: https://docs.solana.com/running-validator/validator-reqs

From the testing I have done, one GPU gets higher throughput, but more than one doesn’t really help much. Check the GPU is actually bound to the process using nvidia-smi.

Believe the main thing is number of cores and clock speed of the cores.

[cjremus]

Submitted results from two setups.

[mbroeken]

Dedicated Root Server SB145
* AMD EPYC 7401P
* 2x SSD U.2 NVMe 960 GB Datacenter
* 4x RAM 32768 MB DDR4 ECC reg.
* NIC 1 Gbit
* - Intel I350
* Location: HEL1

[2020-08-28T09:51:14.265289681Z INFO solana_bench_tps::bench] Node address | Max TPS | Total Transactions
[2020-08-28T09:51:14.265294701Z INFO solana_bench_tps::bench] ---------------------±--------------±-------------------
[2020-08-28T09:51:14.265299221Z INFO solana_bench_tps::bench] 127.0.0.1:8003 | 74717.75 | 4796383
[2020-08-28T09:51:14.265307601Z INFO solana_bench_tps::bench]
Average max TPS: 74717.75, 0 nodes had 0 TPS
[2020-08-28T09:51:14.265313651Z INFO solana_bench_tps::bench]
Highest TPS: 74717.75 sampling period 1s max transactions: 4796383 clients: 1 drop rate: 0.74
[2020-08-28T09:51:14.265320651Z INFO solana_bench_tps::bench] Average TPS: 52885.4