Page 1 of 1

#1 Re: Optimized Apps for Projects

Posted: Fri Jan 25, 2019 8:31 pm
by Bryan
New optimized apps as of 1/13/19 for Acoustics - both Windows and Linux. 2X improvement over the apps the project gives.

The 1st post HERE gives the GitHub link and installation instructions.

#2 Re: Optimized Apps for Projects

Posted: Fri Jan 25, 2019 10:17 pm
by Hal Bregg
Optimisiation for Acoustics@home works well.

Will we see more projects getting similair updates?

#3 Re: Optimized Apps for Projects

Posted: Fri Jan 25, 2019 10:23 pm
by Bryan
It happens sometimes. Usually it is one of the users that produces them and then after some period of time the project will pick those up as the standard apps.

#4 Re: Optimized Apps for Projects

Posted: Sat Jan 26, 2019 4:34 am
by davidbam
Thought I might try this on the 2990WX as the plan is to have that purring along for all sprints. Do I need all 5 of these or only the first 4?
  • acoustic_linux64_avx2_v10.tgz
    acoustic_linux64_avx_v10.tgz
    acoustic_linux64_sse2_v10.tgz
    acoustic_linux64_sse41_v10.tgz
  • acoustic_linux64_avx512_v10.tgz
I tried to work this out based on my flags (below) but still figured it was safer to ask :lol: :lol:

flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht sy
scall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid amd_dcm aperfmperf pn
i pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr
8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx
cpb hw_pstate sme ssbd sev ibpb vmmcall fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec x
getbv1 xsaves clzero irperf xsaveerptr arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pa
usefilter pfthreshold avic v_vmsave_vmload vgif overflow_recov succor smca

#5 Re: Optimized Apps for Projects

Posted: Sat Jan 26, 2019 4:35 am
by Bryan
I played around with the apps tonight. Windows is still 4.5% faster than Linux but a huge improvement. On stock apps Win is 35% faster.

On the 2990WX, running Linux, the AVX2 app is 3X faster than the stock app. It is around 30% faster than SSE4.1 opti app on the 2990WX.

#6 Re: Optimized Apps for Projects

Posted: Sat Jan 26, 2019 4:36 am
by Bryan
You only need the 1st one ... AVX2 for the 2990WX or your Intel machines that support AVX2.

#7 Re: Optimized Apps for Projects

Posted: Sat Jan 26, 2019 4:38 am
by Bryan
Unpack the file and copy the 4 files (that were in the tarball) to your Acoustics project folder. STOP BOINC and then RESTART BOINC and you are good to go. Even if you are already running WU they will finish using the opti app.

#8 Re: Optimized Apps for Projects

Posted: Sat Jan 26, 2019 4:40 am
by davidbam
Glad I asked - many thanks

#9 Re: Optimized Apps for Projects

Posted: Sat Jan 26, 2019 5:20 am
by davidbam
Run time is 25min / 1500secs (give or take a few secs) on 2990WX 3GHz for SSPEMDD v0.18 x86_64-pc-linux-gnu. Temperature stable (yes, I still watch this carefully)

56 pending but 8 computational errors at end of run. Might this be caused by too many jobs finishing at the same time?

#10 Re: Optimized Apps for Projects

Posted: Sat Jan 26, 2019 5:21 am
by Bryan
davidBAM wrote: Sat Jan 26, 2019 5:20 am
56 pending but 8 computational errors at end of run. Might this be caused by too many jobs finishing at the same time?

Yep! I'm surprised there weren't more than that ... I had 36 my 1st load and then I spread them out and haven't seen anymore errors.

#11 Re: Optimized Apps for Projects

Posted: Sat Jan 26, 2019 5:23 am
by davidbam
They are a bit naughty with the deadlines they issue on new WU. I make it that some of them have passed the deadline before they get downloaded

#12 Re: Optimized Apps for Projects

Posted: Sat Jan 26, 2019 5:26 am
by Bryan
Are you seeing some WU that are going into panic mode? I have that on a couple of machines but it will correct itself before long. When a new WU starts up BOINC thinks it will take 2 hours to complete and on those machines they take about 30 minutes.

#13 Re: Optimized Apps for Projects

Posted: Sat Jan 26, 2019 5:29 am
by davidbam
Not sure TBH, I wondered if it was just a way to get run priority from the boinc_client. Mainly I mentioned it to see if it affected bunkering for sprints

#14 Re: Optimized Apps for Projects

Posted: Sat Jan 26, 2019 5:34 am
by davidbam
So I make that about 150K per day for a 2990WX / 3GHz / Linux on Acoustics

#15 Re: Optimized Apps for Projects

Posted: Sat Jan 26, 2019 5:36 am
by Bryan
If you are using BoincTasks it shows the deadline. There are some projects that will throw out some short deadline WU ... WCG for example. If you are bunkering WCG you need to make sure that all WU you are going to cache have a long enough deadline otherwise they time out while you have the lid on it and the machine becomes "untrusted". When you are ready go to "no new work" and run those shorties and turn them in. They you can put the lid on it.

I didn't see any short Acoustics WU on my machines. What is happening is BOINC doesn't really know how long it will take to finish a WU so it makes a guess. As you complete more and more WU it will figure it out and adjust itself and you will stop going into high-priority mode.

#16 Re: Optimized Apps for Projects

Posted: Sat Jan 26, 2019 5:39 am
by davidbam
I'll let it finish the WCG backlog and then try a 4 day bunker for Acoustics

#17 Re: Optimized Apps for Projects

Posted: Sat Jan 26, 2019 5:44 am
by Bryan
Go ahead and load up the machine with Acoustics WU. It will give you 1000 and they have a 7 day deadline. That way they will be aged and start validating almost immediately when you get around to them. Right now I'm only at a 41% validation rate. During the last challenge I found that it takes 24-36 hour before pendings stabilize.

#18 Re: Optimized Apps for Projects

Posted: Sat Jan 26, 2019 2:39 pm
by davidbam
davidBAM wrote: Sat Jan 26, 2019 5:23 am They are a bit naughty with the deadlines they issue on new WU. I make it that some of them have passed the deadline before they get downloaded
Total nonsense. I got the date wrong by a week. In mitigation, it was 5am when I formed that opinion :oops:

#19 Re: Optimized Apps for Projects

Posted: Sat Jan 26, 2019 8:12 pm
by Hal Bregg
What about tweaking amount of WU stored? I read somewhere that if you set those values too hight you might end up in WU not finishing on time.

Speaking of optimalization I went for fastets one based results.

#20 Re: Optimized Apps for Projects

Posted: Sat Jan 26, 2019 8:46 pm
by Bryan
Unless you are bunkered (network communication disabled) then it isn't a problem. If the project sees a WU is past the deadline but it is running on the computer it is okay. Those that go past the deadline and haven't started will get cancelled and removed from your computer by the project.

If you are bunkered and the WU goes beyond the deadline it may or may not get counted and give you credit. When a WU goes past the deadline the project will reissue that to someone else. Most (but not all) projects will give you credit for a late WU if it is turned in before the resend is sent back. There are some projects that will not give you any credit if you are over the deadline.

On projects like Acoustics that have a huge amount of "pendings" it is best to load up as many WU as you can. That way when you start crunching a WU it has "aged" and will and validate sooner.

#21 Re: Optimized Apps for Projects

Posted: Fri Feb 08, 2019 8:56 am
by davidbam
Bryan wrote: Sat Jan 26, 2019 4:36 am You only need the 1st one ... AVX2 for the 2990WX or your Intel machines that support AVX2.
Do you happen to know if these apps will work on a FreeBSD machine? I have a 44 thread Xeon e5 v4 which definitely has AVX2

#22 Re: Optimized Apps for Projects

Posted: Fri Feb 08, 2019 9:39 am
by davidbam
Looks like it does - there is an ABI in place for Linux compatibility for other projects. I will report once I get some validated WU

#23 Re: Optimized Apps for Projects

Posted: Sun Feb 17, 2019 4:04 pm
by davidbam
E5-2690 v2 Xeons don't have AVX2, is AVX the next-best optimisation to use please? Flags at end of lscpu output

Code: Select all

$ lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              40
On-line CPU(s) list: 0-39
Thread(s) per core:  2
Core(s) per socket:  10
Socket(s):           2
NUMA node(s):        2
Vendor ID:           GenuineIntel
CPU family:          6
Model:               62
Model name:          Intel(R) Xeon(R) CPU E5-2690 v2 @ 3.00GHz
Stepping:            4
CPU MHz:             3292.020
CPU max MHz:         3600.0000
CPU min MHz:         1200.0000
BogoMIPS:            5985.49
Virtualisation:      VT-x
L1d cache:           32K
L1i cache:           32K
L2 cache:            256K
L3 cache:            25600K
NUMA node0 CPU(s):   0-9,20-29
NUMA node1 CPU(s):   10-19,30-39
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm cpuid_fault epb pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts flush_l1d

#24 Re: Optimized Apps for Projects

Posted: Sun Feb 17, 2019 7:44 pm
by Bryan
Theoretically it should be. However, if the 2690 downclocks when running AVX then the SSE3 might actually be faster.

#25 Re: Optimized Apps for Projects

Posted: Sun Feb 17, 2019 7:46 pm
by davidbam
2264 secs on average for first 20 WU with AVX. None have yet validated

#26 Re: Optimized Apps for Projects

Posted: Sun Feb 17, 2019 8:08 pm
by davidbam
Fairly hot for a Xeon but can't say I've ever checked it when running WCG. The temperature difference between the two CPUs is quite normal in a Dell as the exhaust from first CPU fan is intake for the second

Code: Select all

Package id 0:  +73.0°C  (high = +81.0°C, crit = +95.0°C)
Core 0:        +70.0°C  (high = +81.0°C, crit = +95.0°C)
Core 1:        +69.0°C  (high = +81.0°C, crit = +95.0°C)
Core 2:        +71.0°C  (high = +81.0°C, crit = +95.0°C)
Core 3:        +71.0°C  (high = +81.0°C, crit = +95.0°C)
Core 4:        +73.0°C  (high = +81.0°C, crit = +95.0°C)
Core 8:        +69.0°C  (high = +81.0°C, crit = +95.0°C)
Core 9:        +70.0°C  (high = +81.0°C, crit = +95.0°C)
Core 10:       +70.0°C  (high = +81.0°C, crit = +95.0°C)
Core 11:       +70.0°C  (high = +81.0°C, crit = +95.0°C)
Core 12:       +69.0°C  (high = +81.0°C, crit = +95.0°C)

Package id 1:  +80.0°C  (high = +81.0°C, crit = +95.0°C)
Core 0:        +80.0°C  (high = +81.0°C, crit = +95.0°C)
Core 1:        +78.0°C  (high = +81.0°C, crit = +95.0°C)
Core 2:        +78.0°C  (high = +81.0°C, crit = +95.0°C)
Core 3:        +80.0°C  (high = +81.0°C, crit = +95.0°C)
Core 4:        +76.0°C  (high = +81.0°C, crit = +95.0°C)
Core 8:        +75.0°C  (high = +81.0°C, crit = +95.0°C)
Core 9:        +80.0°C  (high = +81.0°C, crit = +95.0°C)
Core 10:       +79.0°C  (high = +81.0°C, crit = +95.0°C)
Core 11:       +81.0°C  (high = +81.0°C, crit = +95.0°C)
Core 12:       +78.0°C  (high = +81.0°C, crit = +95.0°C)

#27 Re: Optimized Apps for Projects

Posted: Sun Feb 17, 2019 10:55 pm
by Bryan
The V3 and V4s have AVX2 and they down clock when running AVX2 due to the temperature rise. The V3s have a bug that causes all cores to down clock if any of them are on AVX2. The V4s will only down clock the cores that are running AVX2.