Data analysis is all about statistics. There is your measurement and you have to decide if you get a valid value from your apparatus or is it all noise and nonsense. I don't want to give you a full lecture in it (google for your favourite university + geiger counter experiment + poisson process) but just give you a feel of the limits.

So I have taken 5000 datapoints from Dagorath (cleaned a few false points up), that date back to March 31th. So the samples (they call it ticks here) are from 4 days. First I plotted a histogramm
You can see, that there is a huge span in the detected clicks, there are a few samples with just 2 or over 24 counts, but the majority are about 11 counts. You can fit it with an poisson distribution her, nothing special.

Now a counting experiment relays on (as it is named) counts. The standard devitation, the error of your expected value, is the squareroot of your counts. If you have a poisson process you can't measure it better, you need more counts to get a lesser percentage error.

Look at
.
I have summed every 5 datapoints up, so the counting time is over 3 minutes. The two bands of sigma denotes the measuring error of the mean value (actually every datapoint have got a error bar of sigma). Do you see anything in this noise?

Ok, then take 10 samples

Thats a measuring time of over 6 minutes. Do you see anything?

Next 20 samples

That's a measuring time of over 12 minutes. Well there are not so many spikes there. If you look at the sigma band it is getting thinner too. But because of the square root you need 4 times more counts to get half an error.

Ok, next 40 samples together

That's a measuring time of over 25 minutes. It looks like a steady value with a little noise. Double it again?


Now I summed up over 51 minutes. It looks like the measuring is declining and on the left there is a little hill. But just sum the counts up is a little boring. But we can use a moving average. So we have got a window of lets say 11 values and we shift it over the summed values of eg 40 datapoints. So we preserve a little bit of dynamics and see if the values change in time.

Ok, now have a look at it


The virtual measuring time per point is 282 minutes. That is interesting! Notice that I have rescaled the axis. So the hill is out of the error band, and the measurements are declining to the right. On the beginning of April you got a reading little above background. But you don't know it is radioactivity. Maybe the sun was shining on your detector? Have you taken it to another place, from the cellar to the attic?... Or were there rainy days, it started to get hot and now there is a rainfall again (the reading is climbing again at the end of the graph).

So looking at every click of the detector is like staring on a pixel of your tv set. It is flashing all the time but you do not get the big picture and miss the movie.

:)

background statistics is a complicated topic, as you are never sure where the counts are coming from. To do a correct background measurement is far more complicated than doing measurements of a radioactive sample.
just an example - leaning over the detector automatically should increase the background counts as the K-40 decay inside the body of each of us gives automatically more counts. The crucial question is - how much does each effect (radon emanation after rain, solar modulation etc.) change the mean background value?

i guess that measuring real radioactive samples with our detector will decrease the uncertainty significantly. if desired, i may provide you with the data.

You are right :) And you are not :)

In other topic, I show how to create chart with averages for last 24h. It's very smooth and accurate on your point of view.



But, as TJM mentioned, we will to provide as much data as possible (and as far as it is sensible) and data resolution(?) play a role on it.

Also you can imagine situation of nuclear explosion somewhere near detector - our hardware shows it immediately (well - if it not be destroyed/network unavailable/etc). In this case gamma radiation jump from e.g. 0.16uSv/h to 1000uSv/h (or even more) and differences between 0.09-0.30 not make any difference - you get right signal to run ASAP and from map you get

What is interesting, by analyse data form our project and compare it with other factors (like e.g. dependency type day/night or good/bad weather) you can find more interesting info! This shows that more data equal more possibility to get "something" interesting from it.

Why not use some data as "white noise"? This is another possibility as some people use geiger detectors as random number generator (well, I can imagine other, better methods but here are data available right now).

I know, that our detectors are not laboratory stuff but still collect correct and (I hope) useful data.

@Krzysztof

Can you point me to the thread of the script? The script is not on the radioactiveathome server and just accepts your hostid. It's got to know at least you are working on a script, that is in my opinion more useful.

Please look at the histogramm again. There is a small, but finite probability that you detect eg 26 clicks per trickle. So how can you decide, if there is something going on or it is just the poisson tail of your background? Just by measuring more or a longer time! So it is everytime a tradeoff between accuracy and response to varying conditions. You can never get them both. Two moving averages with different window sizes could do the trick, so you switch in an dynamic mode. But you need to know what do you want - background monitoring then you need data analysis, dosis measurement another device, gamma ray bursts a few hundred millions for a satelite...the famous random number generator-well in the new Intel Ivy Bridge you get one buildin that passes all tests and is no longer pseudorandom. Please collect every sample you want, but please give the users the tools to make reasonable things.

More important would be a local user database, so every client keeps a copy of his records (dating months back and are not erased by monthly server cleaning), that third party tools are able to do nice plots on the users pc and display things in real time.

@Dagorath

I think you are a bit to harsh. I don't wanted to blame anyone. I just wanted to push the team in a more scientific direction. But it is a hobbyist project. I appreciate it. I just stumbled over it because I wanted to build a counter on my own, and so I digged a little bit deeper than the average user. If there are allready over 1000 detectors out I think the team did a good job. The community just have to make it better to suit theire needs.

@ exsafs

background statistics is indeed a complicated topic. I do not say that the slight variations in the analyzed samples are from radiation at all, probably they are temperatur or air preassure effects. Geiger counters are not so senitive and as you know not really used for background in practice. However, you are the only guy with real measuring equipment here. So just the problems in short form and maybe you can help:

The tube STS-5 is a predecessor of the SBM-20 and no real comparisons (besides TJM 5%) are known. The SBM-20 datasheet gives a value of 1 cps for inherit background from the tube alone, that is 3 times the rate from the "background" measurement of my figures. So can you put the device without any radioactive source for a longer time in a
lead castle to get a measurement of the own background? For later data analysis the boinc client needs to run so I don't know how to handle this, maybe put the labtop in the lead chamber too?

Second just 3 more calibration points with Cs137 (under 1mSv/h ... I hope nobody will ever measure such activity) would be nice, so you can draw a least-square line. For the dosis all the self made geiger counter rely on obscure conversion factors. From the datasheet you get a number but nobody knows if it corresbonds to reality. The current recipe is a factor of 171 but there are others like counts in 56 seconds for a reading in mikroRoentgen...

If the overall accuracy of the detecter could be increased by a factor of 2 I think everyone is happy. Just to give an impression what professional equipment is capable of (and doing with your taxes):


You see it started to rain 2 times. The national sensor networks are well developed in central european, so you have a grid to calibrate against it. (And if you find even simular readings you should be really happy).

Links for the national radiation monitoring networks for background comparison

EURDEP covering whole europe but a 6 hour delay


AT
BE
CH
DE
FR
NL
US

Hi,
have to also add some info:

@phys: using Scidavis or Qtiplot aren't you? :-)

We also discussed the measuring time. Is is clear, that longer sampling time would be better for the tube to collect more counts and to get less variable (=more "stable") background measurements. Would be also smaller load for the server etc.

I did some analyses and currently continue with it and we will see.

From the other point of view - this is not scientific measuring equipment for high precision data acquisition but rather an indicator. Ok, maybe using 5-10 samples moving average would improve the values, although wiping out some background variations.

So for example - variations from 100 to 200 nSv/h are relatively big, but it is still background. If there were any radiation event, the GM tube would have enough counts per selected sampling time and would be quite enough for measuring.

Finally, the decision what to do in such situation cannot be based few measurements (could be false points) and on these sensors only - it is just for reference. But, as we know from Japan, such network can help the first respond teams to refine the overview of the situation for next steps.

Very nice! Good work!

____________

From what I see so far on this blog, the detector overestimates the low, background radiation some (under 50%?) while it underestimates high levels by a factor of 2.

This is not necessarily bad as I saw evidence that the esteemed Terra's MKS-05 can underestimate doses 5 fold!

However, since the project wants to monitor the low levels, is it possible to improve results by subtracting the GM tube's own noise of 1 cps or by some other constant adjustments?

Note sure how the adjustments would be deployed across the different hardware versions though.

Really?
You just showed a compelling difference in your above post on Sept 17th.
You can also compare the readings on your map with government network readings also available online.

I'm not talking about #506.
I'm talking about this:


vs that:


As for the BOINC map, I'm looking at 24hrs averages and not instant. I would not call 24 hrs "a very short period of time."

Hi TJM,
we also wanted to recommend to increase the sampling time. Not only because of the database, but also for improving the measurement quality. In our sensor networks we mostly use 10 minute sampling interval which seems to be the best compromise for stationary measuring stations.

I played a little bit with the radioactive@home data from some selected stations and longer sampling interval decreases the value fluctuations very much.

I'm now getting a much 'smoother' graph. Note the last 500 samples as of time of posting this:



This is v1.76, which I guess is the revision that you mention.

Edit: I see other peoples graphs are diffferent too!

Cheers,

Al.
____________

Is is clear, that longer sampling time would be better for the tube to collect more counts and to get less variable (=more "stable") background measurements. Would be also smaller load for the server etc.

That would require to completely redesign the client, server and hardware (sensor)

which were not designed for any kind of real-time operation.

Also, there is just no way our server could handle data at single pulse level. Network is not a problem, but the database was already extremely large with 40s samples. To register each single pulse we would probably need a datacenter, not a single server.

The 'realtime' database size is usually a few gigs for the last 30 days. It has to keep lots of data - each sample has it's geolocation (if present), datestamp, sensor hardware + software id, and user and host info stored.

In the past

we used 40s sample time and it was increased mostly due to severe performance issues. At 40s sample_time a simple script that draws dots on the map was running very slow, some other things started to fail due to timeouts and stressing the hardware too much.

Anyway, our sensors do not support reporting single pulses at all, what they return is raw counter value + hardware timestamp.

Also the entire 'network' we use is not realtime, there is no way to synchronise the clients

so I doubt that even 1s precision would be of any use.

Hi Jason,
not sure if I fully understand what you mean. However, there is it not averaging - you always have pulse count for certain time interval:

sample_ID,host_ID,pulse_count,date_time,Y_coord,x_coord,sample_time,type,experimental,version
45032963,1934,9,2012-06-29 14:16:47,50.037849,14.482327,0.721,n,0,594
45037617,1934,15,2012-06-29 14:17:29,50.037849,14.482327,0.702,n,0,594
45037618,1934,18,2012-06-29 14:18:11,50.037849,14.482327,0.697,n,0,594
45037619,1934,18,2012-06-29 14:18:52,50.037849,14.482327,0.694,n,0,594

Apart from the mentioned data traffic reason there is also another one. As I know, this network was built to provide independent and volunteer radiation monitoring and warning system (although in case of some alarm from such network you should contact institutions responsible for radiation protection or police/fire fighters first).

If you use short time interval to calculate the dose rate (usually microGrays or microSieverts per hour) with low sensitivity small geiger tubes (the SBM-20 have lower sensitivity to background values), then your dose rate values varies much - you can see it on older data. If you collect counts for too short period you have too less data to calculate the dose rate "reasonably accurately". And be sure, that most radiation warning sensor networks use 10 minutes and longer intervals not because of the data traffic.

Finally, I do not think there is a reason to have recorded each count + time value. Using 4 minutes interval to collect counts and also to calculate the dose rate can reduce false trends in data.

Hi
How can i get these software for plot data?
73 de PY2RPD
Wagner
Brazil