12/8-18/8 Merlyn's Memorial-Stage 2 Collatz TSTB Challenge

Alez · Post by **Alez** » Fri Aug 02, 2013 10:48 pm

http://boincstats.com/en/stats/challenge/team/chat/412

Name Merlyn's Memorial - Stage 2
Status Upcoming
Project Collatz Conjecture
Issued by The Scottish Boinc Team
Start time 2013-08-12 01:00 UTC
End time 2013-08-18 01:00 UTC
Late entrants allowed? Yes
Number of teams participating Â» 6
Number of users participating 0

PinkPenguin · Post by **PinkPenguin** » Fri Aug 09, 2013 7:39 am

For those of you returning to collatz after a while (like me) or new to collatz it might be a good idea to read this thread about the SOLO_COLLATZ app:
http://boinc.thesonntags.com/collatz/fo ... =997#16284

Not requiring a wingman is nice... but you do need a bit more CPU to handle it.

Alez · Post by **Alez** » Sat Aug 10, 2013 1:04 am

Think the solo units pay the best as well. 24 hrs to go, time to start stocking up on those units for the big dump.

Post by **skywalker tsbt** » Sat Aug 10, 2013 6:19 pm

Ok.

Had to go to the lab today ... and have all 16 Quad Core PC's with GPU running Collatz.

Got a problematic software installation going on, so they have to be on so I can remote in and fix them. When I figure it out...

Up side, Buko credits!!

Downside, can't think of any.

Alez · Post by **Alez** » Sun Aug 11, 2013 12:34 am

Don't figure it out too quick

Post by **skywalker tsbt** » Sun Aug 11, 2013 12:53 am

alezevo1 wrote:Don't figure it out too quick

LOL. Should prove to be interesting. I've only had at most 4 on in the lab so far. There's 2 more outside of the lab going too. so that's 17 GPU's and whatever 17*4 is? + the 11 cores and 3 GPU's at the homestead.

Feels like I'm getting my Naval Armada ready poised with Air Support.

Alez · Post by **Alez** » Sun Aug 11, 2013 12:59 am

Sounds awesome.
So far I've only been able to get 1 gpu over to collatz, boinc scheduler at its finest as usual

Alez · Post by **Alez** » Sun Aug 11, 2013 2:31 am

ok, very confused, #1 finished...#2 starts in 24 hrs ?

Post by **skywalker tsbt** » Sun Aug 11, 2013 3:07 am

alezevo1 wrote:ok, very confused, #1 finished...#2 starts in 24 hrs ?

:banghead:

Post by **Megacruncher** » Sun Aug 11, 2013 3:15 am

Skywalker TSBT wrote:
alezevo1 wrote:ok, very confused, #1 finished...#2 starts in 24 hrs ?
:banghead:

Yup I got caught out with that too. Stayed up late, a little tipsy but thankfully no Red Bull, uploaded a lot of WU (thankfully not all), checked stats and WTF?

Still, no harm done, lots of WU pending, lots more downloaded which will be uploaded on Monday morning. Not staying up this late on a school night.

Silver · Post by **Silver** » Sun Aug 11, 2013 11:11 am

Megacruncher wrote:
Skywalker TSBT wrote:
alezevo1 wrote:ok, very confused, #1 finished...#2 starts in 24 hrs ?
:banghead:
Yup I got caught out with that too. Stayed up late, a little tipsy but thankfully no Red Bull, uploaded a lot of WU (thankfully not all), checked stats and WTF?

Still, no harm done, lots of WU pending, lots more downloaded which will be uploaded on Monday morning. Not staying up this late on a school night.

And I thought it was planned like that to allow us to stock up plenty of WU?

Mega - Could you not set up your network usage up to only allow access after 01:01 then it will automatically dump all the stockpilled units when the challenge has started.
That's what I did for the start of the first challenge and it seemed to work ok :)

Post by **Megacruncher** » Sun Aug 11, 2013 12:03 pm

Silver wrote:
And I thought it was planned like that to allow us to stock up plenty of WU?

Mega - Could you not set up your network usage up to only allow access after 01:01 then it will automatically dump all the stockpilled units when the challenge has started.
That's what I did for the start of the first challenge and it seemed to work ok :)

Obviously it was planned like that for that very reason - but not all of realised.

It would have to be 02:01 & I prefer manual switching, I've enough work to last to morning and anyway it's only a few hours later.

I've got a fast aTI refusing to run Collatz without errors. I've put it on Dortgen in stead!

Silver · Post by **Silver** » Sun Aug 11, 2013 2:17 pm

Megacruncher wrote:
It would have to be 02:01 & I prefer manual switching, I've enough work to last to morning and anyway it's only a few hours later.

I've got a fast aTI refusing to run Collatz without errors. I've put it on Dortgen in stead!

Yeah

I should've actually checked the times before posting "advice" like that!
As long as you can cope till the morning. I was struggling to pick up enough work to keep me busy til tomorrow but I did get there eventually :)

With regards to your ATI card - are your drivers up to date? I spent ages trying to work out why my Wu's kept coming up with comp error. Updated the driver and seems to be all good now :)

Alez · Post by **Alez** » Sun Aug 11, 2013 6:31 pm

Silver wrote:
With regards to your ATI card - are your drivers up to date? I spent ages trying to work out why my Wu's kept coming up with comp error. Updated the driver and seems to be all good now :)

With Collatz i'm not sure that you want the latest drivers. I run Cat 12.8 and that works fine. I believe that Cat 13.1 drivers don't work on Collatz at all. Maybe later than that are ok.

Post by **Megacruncher** » Sun Aug 11, 2013 10:34 pm

I've got 13.something drivers installed so I'll try rewinding back to 12.8 and see if that helps. Thanks.

Alez · Post by **Alez** » Sun Aug 11, 2013 11:38 pm

http://boinc.thesonntags.com/collatz/fo ... =977#15986

Collatz info on winding back drivers as apparently going backwards with ATI drivers is a messy business. I know I had to remove everything to do a clean reinstall as simply loading the driver over the newer one like you can with nVidia drivers didn't work.

I also notice that every now and again I seenm to get a unit on the ATI that runs far longer than normal. ie 2 that ran for 6 hours. anyone else had this with the solo collatz ? all the cuda ones seem to run for the same time.

Alez · Post by **Alez** » Mon Aug 12, 2013 12:30 am

1 1/2 hrs to go....

Silver · Post by **Silver** » Mon Aug 12, 2013 12:39 am

alezevo1 wrote:
Silver wrote:
With regards to your ATI card - are your drivers up to date? I spent ages trying to work out why my Wu's kept coming up with comp error. Updated the driver and seems to be all good now :)
With Collatz i'm not sure that you want the latest drivers. I run Cat 12.8 and that works fine. I believe that Cat 13.1 drivers don't work on Collatz at all. Maybe later than that are ok.

My WU's kept failing with a computational error as soon as they started running when I was on Cat 13.1
I've upgraded to Cat 13.4, and so far no problems BUT I only upgraded yesterday and have only uploaded a few units to see how it went. So far looks good.

alezevo1 wrote: I also notice that every now and again I seenm to get a unit on the ATI that runs far longer than normal. ie 2 that ran for 6 hours. anyone else had this with the solo collatz ? all the cuda ones seem to run for the same time.

I haven't noticed any strange runtimes yet but like i said only been running since yesterday

Post by **Megacruncher** » Mon Aug 12, 2013 12:53 am

I think I was on 13.4.

As has been noted removing the newer drivers to install the older ones is messy in a geeky fun sort of way. Anyhoo I finally got back to 12.8: No difference whatsoever! Still, worth a go. The NVidias will have to fight this on their own.

Alez · Post by **Alez** » Mon Aug 12, 2013 12:59 am

Megacruncher wrote:I think I was on 13.4.

As has been noted removing the newer drivers to install the older ones is messy in a geeky fun sort of way. Anyhoo I finally got back to 12.8: No difference whatsoever! Still, worth a go. The NVidias will have to fight this on their own.

Im surprised as I thought my 7970 would be good on collatz but in reality the 660 ti's seem to be far better, so the nVidea's win again

unless of course it's a linux - windoze thing as the nVidia's are on the penguin power puter

milkyway and distrtgen are ATI territory, nVidia seems to rule all else.

Post by **Megacruncher** » Mon Aug 12, 2013 1:26 am

Or maybe I wasn't on 13.4 because it works now for Collatz. :)

Anyhow I realised that all my Dirtgen units were crashing on 12.8 so to cut my losses I "reverted" to 13.4, whereapon I found that I could run both projects without either of them crashing!

:wav:

Alez · Post by **Alez** » Mon Aug 12, 2013 1:38 am

Awesome, maybe clearing everything out and reinstalling 13.4 clean is the answer. ATI driver updates have a lot to learn from nVidia about customer satisfaction and ease of update !!! Being playing with the collatz settings and got the solo units down to @ 20 mins now on the 7970. far better

Alez · Post by **Alez** » Mon Aug 12, 2013 2:24 am

We are off and running. Sici have decided to crash this party and we can't discount the amphibians so I guess we are in for a serious dog fight if we want #1 this time. Get crunching boys and girls.....

Silver · Post by **Silver** » Mon Aug 12, 2013 6:41 pm

alezevo1 wrote: Being playing with the collatz settings and got the solo units down to @ 20 mins now on the 7970. far better

Out of interest what settings are you playing with?
If I could tweak my settings to allow my old card to grind out some bigger numbers I'd be delighted!

Alez · Post by **Alez** » Mon Aug 12, 2013 11:29 pm

http://boinc.thesonntags.com/collatz/fo ... hp?id=1009

Changed the number of threads etc to get my ATI card on windoze to match my nVidia's on linux.
The biggest change I managed was by writing an app_config file to allow solo_collatz units a full 1cpu reserved or some of the units would take up to 6 hrs depending on what was running on the cpu's. Now averaging @25 mins.
The gpu's on linux are flying straight out of the box so left well alone :)

Post by **Megacruncher** » Tue Aug 13, 2013 12:31 am

Go on, tell us how to write & install an app_config file and what sort of things to put in it. :)

Anyhow I doubt if we will catch Sici but we are a fairly convincing second at this stage & some decent placings on the individual front. Good to see a good turnout from the competition.

Alez · Post by **Alez** » Tue Aug 13, 2013 2:34 am

Megacruncher wrote:Go on, tell us how to write & install an app_config file and what sort of things to put in it. :)

Are you taking the micky ?

Got solo_collatz on the ATI 7970 down to 11 mins now . Thats a 300% increase on average from where I started.

I believe the micky is being taken you know.....

just in case you're not I will humor you... :)

go to the project directory, in my case
c:/programData/boinc/projects/boinc.thesonntag.com_collatz
open notepad and copy the following exactly

<app_config>
<app>
<name>solo_collatz</name>
<gpu_versions>
<gpu_usage>1</gpu_usage>
<cpu_usage>1</cpu_usage>
</gpu_versions>
</app>
</app_config>

save this file as app_config.xml
do not save as a txt file or it wont work.

scroll down and find a file called solo_collatz_4.07_windows_x86_64_opencl_ati_100 config file ( or the cuda equivalent )
right click on this file and open with notepad

copy the below into it exactly

verbose=0
items_per_kernel=20
kernels_per_reduction=9
threads=8
sleep=1

save file ( not save as), the next solo unit will start using these settings. this is for my 7970, for slower cards start off at items_per_kernel=15 (14 is the default) and work your way up .

more info here http://67.163.87.64/collatz/forum_thread.php?id=1009

Alez · Post by **Alez** » Tue Aug 13, 2013 2:48 am

For the last 2-3 hours all my units are being marked as valid, 0 credits. think we are all in the same boat !!

Silver · Post by **Silver** » Tue Aug 13, 2013 6:06 pm

alezevo1 wrote:
Megacruncher wrote:Go on, tell us how to write & install an app_config file and what sort of things to put in it. :)
Are you taking the micky ?

Got solo_collatz on the ATI 7970 down to 11 mins now . Thats a 300% increase on average from where I started.

I believe the micky is being taken you know.....
just in case you're not I will humor you... :)

Not sure if Mega was taking the micky but I was going to ask the same thing.
Thanks for taking the time to post all the info, I'm going to try your advice and see if I can squeeze anymore out of my old 5750 :)
What gpu load were you getting before you tweaked the settings? my load is pretty close to 100% anyway so I might be pretty much at the limit of my card

Post by **skywalker tsbt** » Tue Aug 13, 2013 7:39 pm

alezevo1 wrote:For the last 2-3 hours all my units are being marked as valid, 0 credits. think we are all in the same boat !!

They are manually adding the credits in. I have at least 20 myself so more credits to come!

Alez · Post by **Alez** » Tue Aug 13, 2013 7:39 pm

If you are at nearly 100 % load then don't bother or it will simply crash. Before I was getting 60 -65 % load so lots of overhead. Now @ 97 %. Could probably get a touch more with some more overclocking but don't think it worth the effort whilst everything is stable.
If you have a little overhead left then you may get a wee bit more by setting items_per_kernel to 15 but leave kernels_per_reduction=8.
You will probably only get a speed up by doing the first tweak and reserving a whole core for running the gpu as that seems to stabilize the time taken per unit. ( before doing that I was lurching between 33 mins to 6 hours ) Collatz pays on a per step basis, not time spend so the credit is nearly always the same regardless of the time taken.

The more we can push the better as we are being caught by the frenchies...

Post by **Megacruncher** » Tue Aug 13, 2013 7:45 pm

alezevo1 wrote:
Megacruncher wrote:Go on, tell us how to write & install an app_config file and what sort of things to put in it. :)
Are you taking the micky ?

Not really, it was a genuine enquiry and only the gentlest mockery was aimed at your casual, & wildly optimistic assumption, that all the rest of us would know what you were talking about and exactly how to go about it. :)

Thanks anyway for humouring me. Now I know I too will give this a go.

Alez · Post by **Alez** » Tue Aug 13, 2013 7:55 pm

Megacruncher wrote:
alezevo1 wrote:
Megacruncher wrote:Go on, tell us how to write & install an app_config file and what sort of things to put in it. :)
Are you taking the micky ?
Not really, it was a genuine enquiry and only the gentlest mockery was aimed at your casual, & wildly optimistic assumption, that all the rest of us would know what you were talking about and exactly how to go about it. :)

Thanks anyway for humouring me. Now I know I too will give this a go.

I will consider my leg to have been stretched by a gentle inch or two

As you are the 'godfather' I did indeed presume that you could write config files etc.
About to go and attempt this on my linux machine and see if I can push more from the nVidias but suspect they will simply crash as at 26 mins a unit suspect that is the performance that they will give, we will see .....

Alez · Post by **Alez** » Tue Aug 13, 2013 7:58 pm

and a very timely and welcome credit dump by Mega gets the 'auld alliance' off our butts for a bit

Silver · Post by **Silver** » Tue Aug 13, 2013 8:23 pm

alezevo1 wrote:If you are at nearly 100 % load then don't bother or it will simply crash. Before I was getting 60 -65 % load so lots of overhead. Now @ 97 %. Could probably get a touch more with some more overclocking but don't think it worth the effort whilst everything is stable.
If you have a little overhead left then you may get a wee bit more by setting items_per_kernel to 15 but leave kernels_per_reduction=8.
You will probably only get a speed up by doing the first tweak and reserving a whole core for running the gpu as that seems to stabilize the time taken per unit. ( before doing that I was lurching between 33 mins to 6 hours ) Collatz pays on a per step basis, not time spend so the credit is nearly always the same regardless of the time taken.

The more we can push the better as we are being caught by the frenchies...

Thanks again for your advice alezevo1

I think you have just confirmed that I'm pretty much flat out now - which is good in some ways but leaves nowhere to go

I'd already set the boinc manager preferences to use no more than 95% of processor and this usually means my total CPU usage is approx 75% however I see that my 'spare core' is being used by solo collatz to do the verifying thing, so i don't think I'm going to be able to improve much at all

I might have to think about getting a newer faster card now although I seem to be generating my highest daily numbers ever doing collatz

Paul

Alez · Post by **Alez** » Tue Aug 13, 2013 8:40 pm

Silver wrote:[

I might have to think about getting a newer faster card now

Paul

The bug bites.....

There's always room for a faster card, then there's that spare slot, then there's that spare space in the room we don't use..........
The 24 hr service of amazon is a god send and a nightmare....Bwahaahaa.

Alez · Post by **Alez** » Tue Aug 13, 2013 10:15 pm

Managed a 20% increase in productivity on the nVidia cards. Not much but as that's on 3 cards so it will add up. Also shows the Linux version running cuda50 is more efficient out of the box.
I want 2nd place back so now got to try and force the last nVidia to accept collatz work

Alez · Post by **Alez** » Wed Aug 14, 2013 1:03 am

all the 0 credit units have been credited

and I have been knocked down to 4th

just when I nearly had 2nd back.... oh well crunch on.

Alez · Post by **Alez** » Wed Aug 14, 2013 10:16 pm

TSBT running second. No chance of catching Sici unless they stop crunching but we are in a dogfight with the frenchies and will need to keep pushing to stay ahead of them.
Personally have managed to take 2nd back

but still a GPU down on this project. The stupid scheduler refuses to budge and give me units even when the gpu is allowed to sit idle

Silver · Post by **Silver** » Wed Aug 14, 2013 10:55 pm

alezevo1 wrote:Managed a 20% increase in productivity on the nVidia cards. Not much but as that's on 3 cards so it will add up.

20% not much?! That sounds pretty good to me.

Well done taking second spot back

Fingers crossed we can keep ahead of our French chums!

Post by **Megacruncher** » Wed Aug 14, 2013 11:32 pm

alezevo1 wrote:
...
As you are the 'godfather' I did indeed presume that you could write config files etc.

Being godfather means I can arrange for horses' heads to be left in peoples' beds (or even the other way round if I'm really cross) but it doesn't make me an expert on the fine tuning of all boinc projects.

If pressed I can write a config file but I'd be lying if I said that I fully understood them. Brute force & ignorance (along with a shedload of persistence) have got me a long way and unless a recomendation is particularly reliably sourced I tend to run everything on stock settings. I've too many machines and run too many projects to be footering about all the time.

Some of my past dabblings with optimizations have caused me nothing but grief but I don't have any problems with errors with this and the speed increase is about 20%. :)

I've got a lot of Collatz & mini-collatz left to crunch - does this work for them and what settings should I use? The same?

Post by **Megacruncher** » Thu Aug 15, 2013 12:57 am

Right, so just to prove that I can do it when my boinculinity is so aggressively challenged I edited the various config files and the app_config.xml one (slavishly copying over Alezevo1's settings) and established that they seem to work with a 20% gain over all the Collatz gpu projects. Which is nice!

The French will be left behind, or not, as the case may be!

Alez · Post by **Alez** » Thu Aug 15, 2013 1:04 am

Eeek, as I've already got 4 horses here can we miss out the heads in the beds part, unless of course you require a few heads supplied

I am only running the solo units so I don't have to wait for validation. The same settings should work fine as all three applications do exactly the same thing, mini being smaller, collatz being the normal unit and solo being collatz with self validation. All three apps have their own config files and are changed exactly the same way. Again the config is copied to the slot so any changes made will take place when the next unit starts.

If you are using the 680 nVidia card you may be able to push it even further than what I have though whether it's worth trying is up to you. Those setting leave my cards stable, there may be room for some more tweaking but not worth the effort if it involves babysitting them.

Alez · Post by **Alez** » Thu Aug 15, 2013 1:08 am

Megacruncher wrote:Right, so just to prove that I can do it when my boinculinity is so aggressively challenged I edited the various config files and the app_config.xml one (slavishly copying over Alezevo1's settings) and established that they seem to work with a 20% gain over all the Collatz gpu projects. Which is nice!

The French will be left behind, or not, as the case may be!

Does that mean you're going to try for 3rd ?

The French aren't doing that well individually, it's just that there are lots of them.... if a few more joined in for us then things will be different. As it is I think we will hold them off and take 2nd. Think we will miss Janos's output on Milkyway though.

Alez · Post by **Alez** » Thu Aug 15, 2013 1:21 am

Silver wrote:
alezevo1 wrote:Managed a 20% increase in productivity on the nVidia cards. Not much but as that's on 3 cards so it will add up.
20% not much?! That sounds pretty good to me.

I managed 300% with the ATI card

Alez · Post by **Alez** » Thu Aug 15, 2013 6:23 pm

So hot and humid here that the gpu's in Orac are running in the 80's C. had to take the case off, lay Orac on his back and stick a portable fan to blast the GPU's. Much more of this and I will have to shut down

Who could consider this can be the case in Scotland.

Post by **Megacruncher** » Thu Aug 15, 2013 11:14 pm

It is a bit warm. 28degC in the basement with the door wide open.

Anyway Sici will win - even if they ceased this instant and TSBT & L'AF joined forces.

But we won't join them. We will beat them! We are now 2.2Million ahead of the French a gap which is slowly but steadily widening.

On the individual front your second place looks pretty secure. I'm in 5th, just, but I think I can hold it - the next 47 hours will be intense with all GPUs on full-time optimised Collatz (unlike today when one of the GTX580s got distracted by some Einstein WUs reaching their deadline.

There's 17 of us on it in total which is not bad at all.

Thanks everyone! :cheers:

Silver · Post by **Silver** » Fri Aug 16, 2013 12:37 am

alezevo1 wrote:So hot and humid here that the gpu's in Orac are running in the 80's C. had to take the case off, lay Orac on his back and stick a portable fan to blast the GPU's. Much more of this and I will have to shut down Who could consider this can be the case in Scotland.

I know, it's not that often I wish it was a bit cooler!

Silver · Post by **Silver** » Fri Aug 16, 2013 12:47 am

I've also done a little bit of tweaking and a bit of graphics card overclocking seems to have gained me ~15% increase :)
Seems to be stable so far and the first couple of units have been error free so hopefully that will help the teams numbers a bit

Alez · Post by **Alez** » Sat Aug 17, 2013 1:15 am

Had to underclock the ATI card and back off the optimization a bit this afternoon as without the portable cooling fan the gpu was trying to hit 90 C. Zero wind, south facing house, pointless trying to cool equipment with hot air

More fans have been ordered and the case is about to get yet another big hole drilled in it.

We are comfortable in 2nd now with only 24 hrs to go. :)