Page 1 of 2

#1 NumberFields

Posted: Sat Mar 23, 2019 5:20 am
by davidbam
The new nVidia WU mean that I can theoretically score about 3.25 million per day on a single 1080 (provided the project offers enough work). https://tsbt.co.uk/forum/viewtopic.php?f=193&t=11425

I realise that the sprint is newly finished but I am happy to leave a 1080 on this to see if we can catch the US Navy before they wake up to it. If you have CPUs on this, you may want to monitor the situation to see if they are better deployed elsewhere

#2 nVidia GTX1080 GPU on Linux

Posted: Sat Mar 23, 2019 6:00 am
by davidbam
Run TimeCPU TimeCreditPer day/procCredit per kWHApplication
231.85 230.94 8,7653,266,393756,110Get Decic Fields v3.00 (cuda30)
550.85 550.37 20,8253,266,442756,121Get Decic Fields v3.00 (cuda30)
199.73 197.16 7,5513,266,498756,134Get Decic Fields v3.00 (cuda30)
237.82 235.96 8,9913,266,481756,130Get Decic Fields v3.00 (cuda30)
181.68 179.84 6,8683,266,384756,107Get Decic Fields v3.00 (cuda30)
277.97 277.57 10,5093,266,481756,130Get Decic Fields v3.00 (cuda30)
340.18 339.74 12,8613,266,450756,123Get Decic Fields v3.00 (cuda30)
441.64 440.97 16,6973,266,456756,124Get Decic Fields v3.00 (cuda30)
493.75 493.40 18,6673,266,468756,127Get Decic Fields v3.00 (cuda30)
422.47 422.15 15,9723,266,489756,132Get Decic Fields v3.00 (cuda30)
291.01 289.70 11,0023,266,405756,112Get Decic Fields v3.00 (cuda30)
242.87 241.06 9,1823,266,441756,121Get Decic Fields v3.00 (cuda30)
385.43 384.51 14,5723,266,500756,134Get Decic Fields v3.00 (cuda30)
237.91 236.07 8,9953,266,480756,130Get Decic Fields v3.00 (cuda30)
275.91 274.82 10,4313,266,419756,115Get Decic Fields v3.00 (cuda30)
248.87 248.10 9,4093,266,484756,131Get Decic Fields v3.00 (cuda30)
434.43 433.86 16,4243,266,450756,123Get Decic Fields v3.00 (cuda30)
314.17 313.71 11,8783,266,457756,124Get Decic Fields v3.00 (cuda30)
269.91 268.96 10,2043,266,410756,114Get Decic Fields v3.00 (cuda30)
296.13 295.16 11,1963,266,456756,124Get Decic Fields v3.00 (cuda30)
318.73317.7012,0503,266,452756,123Averages for 20 WU on nVidia GTX1080 GPU, 180W, 1 WU per proc

#3 Re: NumberFields

Posted: Sat Mar 23, 2019 6:02 am
by davidbam
first 20 WU above - note that it needs 100% of one core as well. My core wasn't particularly awesome (E5-2697 v2 @ 2.70GHz) and could well be the limiting factor

What is particularly interesting is that, for some of these, I had a GPUGrid job running on the same GPU from a different boinc-client instance (not entirely deliberately, it should be said :oops: ). It seems to make no difference at all to the NumberFields job - time will tell if GPUGrid job is badly hampered. If it is also okay - these are basically 'free' credits

Looking back at the Sprint, as a team, we managed 2M per day but I reckon a single 1080 can easily surpass that figure

#4 Re: NumberFields

Posted: Sat Mar 23, 2019 7:50 am
by davidbam
I now have 4 gtx1080 on it so fully expect us to topple US NAVY for top spot in the marathon today(as long as the WU keep coming)

I've DM'ed the project admin (Eric) and he says he will keep the WU going for nVidia if anyone wants to jump on the bandwagon. (Need Linux and recent drivers - I have 418.43)

#5 Re: NumberFields

Posted: Sat Mar 23, 2019 9:56 am
by davidbam
not getting enough NF work but happily the GPUs work away on GPUGrid regardless.

I'm not sure if project admin is completely aware of how popular this will be - basically renders CPU WU obsolete

#6 Re: NumberFields

Posted: Sun Mar 24, 2019 7:39 am
by Alez
I need to check which drivers I'm on. Claims I need to upgrade to the latest version. Not sure the admin is aware that Linux distros rarely have cutting edge drivers in the repos. Job for later. Shiny stuff to attempt to win ....

#7 Re: NumberFields

Posted: Sun Mar 24, 2019 8:41 am
by davidbam
Looks like Ukraine are the only other League 2 team to have twigged

#8 Re: NumberFields

Posted: Sun Mar 24, 2019 7:00 pm
by Bryan
With the GPU app available this probably doesn't matter anymore but you can double your CPU credits by over-committing resources by 2X. phoenicis, from XS, did that on the last challenge. NF uses "run time" and not CPU time for computing credits. If you over-commit a machine the run time doubles and it doubles your credits/WU. Since you have 2 WU running simultaneously it doubles your daily credits.

phoenicis had one 1950X running 3X and he was getting a 3:1 payout.

I've confirmed that this works.

#9 Re: NumberFields

Posted: Sun Mar 24, 2019 10:31 pm
by davidbam
I guess you'd have to have enough RAM to prevent any swappng. Does the unix scheduler introduce any significant overhead or not?

#10 Re: NumberFields

Posted: Mon Mar 25, 2019 5:12 am
by Bryan
The scheduler adds very little to the time. The run time is very close to 2X the cpu time. You could even run it in several over-committed VMs to get even more credits. That's what we used to do in the WCG time challenges. You would run 3 VM with each setup as a std. VM. Meaning if the machine has 24 threads you run 3 24 thread VMs. As far as WCG was concerned what it saw was the time. Credits weren't any higher because WCG uses run time and benchmarks to determine the credits.

#11 Re: NumberFields

Posted: Mon Mar 25, 2019 5:45 am
by davidbam
There is a bug in v3.00 of the GPU WU. If you have multiple GPUs in one machine all WU will be run on the first GPU rather then one PER gpu. Version 3.01 supposedly fixes thit but I've got lots of 3.0 downloaded to get through first

#12 Re: NumberFields

Posted: Mon Mar 25, 2019 11:32 am
by Bryan
Does anyone know if you can run a GPU in a VM?

#13 Re: NumberFields

Posted: Mon Mar 25, 2019 11:37 am
by davidbam
Pretty sure you can as there are some Data Centres starting to offer that option.

Do I know how to do it? Sorry no :lol:

#14 Re: NumberFields

Posted: Mon Mar 25, 2019 11:54 am
by Bryan
Data Centers don't use something like VBox though. They are usually bare-metal hypervisors. On those you can run a VM within a VM which is something you can't do with VBox. They have much more flexible capability.

#15 Re: NumberFields

Posted: Mon Mar 25, 2019 12:04 pm
by davidbam
I know nothing about VBox, sorry, I only use bhyve VMs under freeBSD (and even then am completely reliant on John to do the difficult stuff). Coincidentally, I asked him about GPU in a VM several months ago and he thought it might need a feature called PCI-passthru.

#16 Re: NumberFields

Posted: Mon Mar 25, 2019 1:17 pm
by Alez
As far as I'm aware you can't run a GPU on VBox as it's not supported. There are no projects that try to do this.
It might be possible but would require PCI Passthrough on Linux. However I doubt the drivers on the system would function correctly and I think you would have great difficulty sharing the GPU with the system and the VM

However it is being discussed so may be a way - https://github.com/BOINC/boinc/issues/2514

t
he thing to do is PCI Passthrough in Linux with KVM. That's not the switch, but Kernel Virtual Machine

VirtualBox does support using a GPU this way, but only on Linux and only with PCI and some PCIe cards.
If you want to see how to set this up for playing games in Windows in a VM running from Linux, read or watch the video on Play games in Windows on Linux PCI Passthrough

There are hardware requirements to do all this.
The major one is the right CPU (and motherboard of course), one that supports VT-d for Intel (list here) and IOMMU/AMD-Vi for AMD (list here)

#17 Re: NumberFields

Posted: Mon Mar 25, 2019 2:10 pm
by Bryan
I think that answered the question Alez ... thanks!
One big problem (also said by Austin): the Nvidia driver is able to detect if it is running in a VM, and if it is, it will shut itself down. This happens for all consumer grade GPUs, but appears not to be present on professional versions (such as the Quadro 2000 and above).

#18 Re: NumberFields

Posted: Mon Mar 25, 2019 2:42 pm
by davidbam
Bryan wrote: Sun Mar 24, 2019 7:00 pm phoenicis had one 1950X running 3X and he was getting a 3:1 payout.
He is doing something rather well as I see he is earning 63 million per day on NumberFields at the moment. Damn - now I'm only #2 on the RAC https://boincstats.com/en/stats/122/user/list/16/0/0

He must have a few GPUs as well methinks :P :P

#19 Re: NumberFields

Posted: Tue Mar 26, 2019 2:19 am
by scole of TSBT
The GPU credits sure add up fast. Better get what you can before they change the credit schema.

#20 Re: NumberFields

Posted: Tue Mar 26, 2019 2:34 am
by Alez
Upgraded to 418.56 and not getting a single GPU unit to crunch.
Looking at the message board appears Eric has set credit screw so boo hoo sucks to him.

#21 Re: NumberFields

Posted: Tue Mar 26, 2019 3:01 am
by scole of TSBT
Alez wrote: Tue Mar 26, 2019 2:34 am Upgraded to 418.56 and not getting a single GPU unit to crunch.
Looking at the message board appears Eric has set credit screw so boo hoo sucks to him.
Do you have Use NVIDIA GPU and Run test applications selected?

#22 Re: NumberFields

Posted: Tue Mar 26, 2019 3:32 am
by Alez
Yes, finally got some to crunch. So far so good, only one errored.

#23 Re: NumberFields

Posted: Tue Mar 26, 2019 6:46 am
by davidbam
I am running 6 WU per gtx1080 and stats still seem to be getting better. Trying 9 WU per GPU today

Yes, Eric has throttled back the credits paid so I will wait a day or two before posting the earnings I think I am getting

#24 Re: NumberFields

Posted: Tue Mar 26, 2019 11:30 am
by scole of TSBT
I was earning 20 credits per run sec. Now I'm only earning .08 credits per run sec, 1/250 of what is was. An extremely poor paying GPU project, similar to Asteroids. They ruined it.

#25 Re: NumberFields

Posted: Tue Mar 26, 2019 11:34 am
by davidbam
Hmmm - I am presently earning about 1-1.2M per hour from only two gtx1080 cards. Is it possibly because they are the v3.00 rather than the v3.01 app?

I am creeping upwards towards 16 WU per card but I'm not there yet as the optimal CPU requirement is proving elusive

#26 Re: NumberFields

Posted: Tue Mar 26, 2019 11:42 am
by davidbam
BTW, click STATS today in the pink box on https://tsbt.co.uk/forum/viewforum.php?f=64 for rapid assessment of how you are doing

#27 Re: NumberFields

Posted: Tue Mar 26, 2019 12:56 pm
by Alez
I'm lucky if I'm getting 12 credits per task now. Not worth running the GPU's on the project anymore.
Credit screw strikes again I'm afraid.

#28 Re: NumberFields

Posted: Tue Mar 26, 2019 1:12 pm
by davidbam
Ah well, the good news is that our lead in the Marathon will now hopefully be unassailable ?

#29 Re: NumberFields

Posted: Tue Mar 26, 2019 1:25 pm
by Alez
Unless the project decides to remove the credit retrospectively, then I would say so. I went and moaned on the project forum that now the credit is so low, it's pointless running the GPU app. I absolutely hate credit screw. The CPU credit will also completely crash to Seti level random credit returns as a result of implementing credit screw.

#30 Re: NumberFields

Posted: Tue Mar 26, 2019 2:31 pm
by davidbam
This app_config will get you about 120K / day from a GTX1080 plus 8 cpu cores. It runs 16 WU simultaneously. I haven't tried more but every reason to suspect it is a linear improvement in payments without actually returning any more completed work. And who thought CreditNew was remotely fair?

Next test is 32 WU on 4 cores

nvidia-smi is a very handy shell command for seeing what is running on multi-GPU systems

Code: Select all

<app_config>
<app>
<name>GetDecics</name>
<max_concurrent>16</max_concurrent>
<gpu_versions>
<gpu_usage>0.0624</gpu_usage>
<cpu_usage>0.5</cpu_usage>
</gpu_versions>
</app>
</app_config>

#31 Re: NumberFields

Posted: Tue Mar 26, 2019 2:59 pm
by davidbam
Bryan wrote: Sun Mar 24, 2019 7:00 pm over-committing resources
Probably the same principle at play with GPU WU

#32 Re: NumberFields

Posted: Tue Mar 26, 2019 3:02 pm
by Bryan
I think he has turned on Credit Screw and the payout went in the toilet.

#33 gtx1080 plus 8 cpu cores, 16 simultaneous WU

Posted: Tue Mar 26, 2019 3:07 pm
by davidbam
This gives approx 108K per day from a single card.
Run TimeCPU TimeCreditPer day/procCredit per kWHApplication
1757.38 1737.63 1075,2781,222Get Decic Fields v3.01 (cuda30)
2555.17 2527.27 2237,5371,745Get Decic Fields v3.01 (cuda30)
3230.75 3177.30 2035,4331,258Get Decic Fields v3.01 (cuda30)
2341.95 2315.23 1465,3781,245Get Decic Fields v3.01 (cuda30)
1720.36 1700.25 1035,1911,202Get Decic Fields v3.01 (cuda30)
2162.04 2123.78 1305,1911,202Get Decic Fields v3.01 (cuda30)
2399.32 2373.12 2097,5371,745Get Decic Fields v3.01 (cuda30)
3255.10 3232.81 1995,2951,226Get Decic Fields v3.01 (cuda30)
2294.17 2263.22 2007,5371,745Get Decic Fields v3.01 (cuda30)
1921.83 1895.72 1175,2781,222Get Decic Fields v3.01 (cuda30)
2041.34 2014.38 1787,5351,744Get Decic Fields v3.01 (cuda30)
1746.51 1714.90 1527,5441,746Get Decic Fields v3.01 (cuda30)
1296.49 1279.34 1137,5441,746Get Decic Fields v3.01 (cuda30)
3047.70 3020.47 2667,5351,744Get Decic Fields v3.01 (cuda30)
1655.65 1635.68 1457,5441,746Get Decic Fields v3.01 (cuda30)
2680.80 2657.81 2347,5371,745Get Decic Fields v3.01 (cuda30)
2864.04 2812.72 2507,5351,744Get Decic Fields v3.01 (cuda30)
2033.26 2022.87 1787,5441,746Get Decic Fields v3.01 (cuda30)
2409.51 2386.14 2107,5371,745Get Decic Fields v3.01 (cuda30)
2810.38 2796.46 2457,5351,744Get Decic Fields v3.01 (cuda30)
2,311.192,284.361806,752 * 161,563 * 16Averages for nVidia GTX1080 GPU, 180W, 16 WU per proc

#34 Re: NumberFields

Posted: Tue Mar 26, 2019 3:36 pm
by Bryan
What a waste of a good GPU!

#35 Re: NumberFields

Posted: Tue Mar 26, 2019 3:40 pm
by davidbam
Yes, I think Eric has been persuaded by folks with hard-won CPU credit totals.

32WU doesn't work BTW - probably because it needs more RAM than the card has. Now trying 16WU and only 4 cpu threads

#36 Re: NumberFields

Posted: Tue Mar 26, 2019 3:42 pm
by Alez
Eric did implement credit screw and the payout crashed to being not worth the effort.
He has now changed it again to a fixed credit. I just got 4000 for a task. Better than 12 I suppose. Least he seems to have listened to our complaints.

#37 Re: NumberFields

Posted: Tue Mar 26, 2019 4:12 pm
by davidbam
ah ha - probably invalidates my 16WU technique (which is fair enough as it was a con)

#38 Re: NumberFields

Posted: Tue Mar 26, 2019 4:14 pm
by davidbam
and my sky-high returns on v3.00 WU have also stopped. Ah well, it was good while it lasted.

They ought really to restart the FB Marathon but I kinda hope they don't

#39 Re: NumberFields

Posted: Tue Mar 26, 2019 4:47 pm
by scole of TSBT
He has now set the credit to 4k per WU.

https://numberfields.asu.edu/NumberFiel ... =2294#2294

#40 Re: NumberFields

Posted: Tue Mar 26, 2019 5:16 pm
by Alez
Better. My RAC has started rising again after the credit screw free fall.

#41 Re: NumberFields

Posted: Tue Mar 26, 2019 7:55 pm
by scole of TSBT
davidBAM wrote: Tue Mar 26, 2019 11:42 am BTW, click STATS today in the pink box on https://tsbt.co.uk/forum/viewforum.php?f=64 for rapid assessment of how you are doing
I don't see the Stats Today in the pink box on that page :(

#42 Re: NumberFields

Posted: Tue Mar 26, 2019 8:30 pm
by davidbam
They are links even if they don't look like them. Output appears in pink box. I'm at rehearsal but can post a screen grab when I get home.

#43 Re: NumberFields

Posted: Tue Mar 26, 2019 9:57 pm
by davidbam
Can one of you post a screen dump of the top of the page please

#44 Re: NumberFields

Posted: Tue Mar 26, 2019 10:44 pm
by scole of TSBT
davidBAM wrote: Tue Mar 26, 2019 9:57 pm Can one of you post a screen dump of the top of the page please
See attached files

#45 Re: NumberFields

Posted: Tue Mar 26, 2019 11:22 pm
by davidbam
Dear dear dear - I must be getting old. I had forgotten one key part in making this work for all TSBT group members.

Sorry about that Gents - you should see a heckuva difference now since almost 20 other project forums had the same problem. Hopefully all sorted now

#46 Re: NumberFields

Posted: Wed Mar 27, 2019 1:30 am
by Alez
Think we should move this topic to the project forum. Also my 1070 and 1060 on this are fairing quite well now.

#47 Re: NumberFields

Posted: Wed Mar 27, 2019 4:12 am
by davidbam
Alez wrote: Wed Mar 27, 2019 1:30 am Think we should move this topic to the project forum. Also my 1070 and 1060 on this are fairing quite well now.
I'm never exactly sure which techniques we discuss in open forums vs. what we discuss in 'pirates' forums TBH.

That said, I reckon deleting this particular thread wouldn't be a huge loss since it is pretty irrelevant after two separate adjustments to the way project points are awarded !! Up to you, I don't mind either way

#48 Re: NumberFields

Posted: Wed Mar 27, 2019 11:37 am
by Alez
I don't think there is anything in the thread we don't want the world to see and it's good to let people know what sort of work and discussion goes on in the background of the team.

#49 Re: NumberFields

Posted: Wed Mar 27, 2019 11:41 pm
by Alez

#50 Re: NumberFields

Posted: Wed Mar 27, 2019 11:43 pm
by davidbam
I changed your date format :P