Data logging in offline mode


Advanced search

Message boards : Science : Data logging in offline mode

Author Message
Profile jhelebrant
Avatar
Send message
Joined: 30 Jul 12
Posts: 27
Credit: 1,521
RAC: 0
Message 2368 - Posted: 21 Feb 2014 | 8:42:41 UTC

hi,
we would like to test the detector in our gamma and RTG irradiator in our institute but there is no internet connection available. Is there any known way to log the data from sensor in a file or something else? Could be in terminal - linux or windows - have both.

I am currently thinking about sniffing the using HHD free serial port monitor as the sensor uses virtual COM port via usb as I know. Will try it today but maybe there is a better way.

thanks

phys
Avatar
Send message
Joined: 28 Apr 12
Posts: 24
Credit: 0
RAC: 0
Message 2385 - Posted: 27 Feb 2014 | 20:49:53 UTC

It is a shame none of the devs answered your questions. I want to give you some hints:

The device uses a HID interface. A free tool to communicate with HID devices is HIDAPI, also available as phyton module, so you may use certain commands on the command line without making a whole C-program. Firstly, you must get the VID and PID of your device, eg for v3 Detector: 0x04d8,0xf6fe.

The correct data transfer mode is "Feature report". Not Interrupt in or interrupt out. The report ID is 0. This is a special ID, the only bidirectional one. The report size is 0x80 or 128 (probably 129?). If you get the feature report, you will see two changing fields over several bytes. One is the time in milliseconds, the other one the detected counts.

To send commands to the device, you put the control command on the first place in the feature report and send the buffer to the HID.

The control commands are
#define GM_SUPPLY_ON 0xa1
#define GM_SUPPLY_OFF 0xa0
#define RESET_COUNTERS 0xa2
#define LCD_BACKLIGHT_ON 0x11
#define LCD_BACKLIGHT_OFF 0x10
#define BUZZER_ON 0x41
#define BUZZER_OFF 0x40
#define BUZZER_ALERT 0x42 //switch on the buzzer on reading >0.3
#define LCD_OPT_0 0x50 //old-style LCD update - 35s
#define LCD_OPT_1 0x51 //running average, display updated every 3s
#define LCD_OPT_2 0x52 //350s average

Happy hacking! And please post your code.

Profile krzyszp
Project administrator
Project developer
Project tester
Project scientist
Avatar
Send message
Joined: 16 Apr 11
Posts: 373
Credit: 462,192
RAC: 135

Message 2386 - Posted: 28 Feb 2014 | 0:09:47 UTC - in response to Message 2385.

All sources are published...
Everybody can look on it ;)
This is best example...

I know, that is now 'wiki' or something like that, but...
____________
Regards,
Krzysztof 'krzyszp' Piszczek
Android Radioactive@Home Map
Android Radioactive@Home Map - donated
My Workplace

phys
Avatar
Send message
Joined: 28 Apr 12
Posts: 24
Credit: 0
RAC: 0
Message 2387 - Posted: 28 Feb 2014 | 1:59:30 UTC - in response to Message 2386.

I know, that is now 'wiki' or something like that, but...


Yeah, there is no real documentation. You have to find it between the lines. And sniff the USB communication

<entry Type="URB" StartEnd="Start" Request="CLASS_INTERFACE" IO="IN" ElapsedSeconds="760.590980" DeviceObject="FFFFFA80065E5060" DeviceObjectName="\Device\USBPDO-5" Irp="FFFFFA800C88DA20" Status="STATUS_SUCCESS" Process="radac_1.77_win">
<IRP>
<URB>
<UrbControlVendorClassRequest>
<Hdr>
<Length>136</Length>
<Function>1b</Function>
<Status>USBD_STATUS_SUCCESS</Status>
<UsbdDeviceHandle>0000000000000000</UsbdDeviceHandle>
<UsbdFlags>0</UsbdFlags>
<Raw>88 00 1B 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 80 00 00 00 05 09 BD 0A 80 F8 FF FF 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 22 01 00 03 00 00 00 00</Raw>
</Hdr>

And the device answers with

<entry Type="IRP" StartEnd="End" Request="IRP_MJ_INTERNAL_DEVICE_CONTROL" IO="--" ElapsedSeconds="760.590980" DeviceObject="FFFFFA80065E5060" DeviceObjectName="\Device\USBPDO-5"
<TransferFlags>TRANSFER_DIRECTION_IN</TransferFlags>
<TransferBufferLength>80</TransferBufferLength>
<TransferBuffer>FFFFF8800360B905</TransferBuffer>
<TransferBufferData>FC A8 2F 03 D1 62 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00</TransferBufferData>


This is not self-explanatory!
So I inspected the AVR microcontroller source. Here I got in main.c the answers

PROGMEM char usbHidReportDescriptor[22] = { /* USB report descriptor */
0x06, 0x00, 0xff, // USAGE_PAGE (Generic Desktop)
0x09, 0x01, // USAGE (Vendor Usage 1)
0xa1, 0x01, // COLLECTION (Application)
0x15, 0x00, // LOGICAL_MINIMUM (0)
0x26, 0xff, 0x00, // LOGICAL_MAXIMUM (255)
0x75, 0x08, // REPORT_SIZE (8)
0x95, 0x80, // REPORT_COUNT (128)=0x80
0x09, 0x00, // USAGE (Undefined)
0xb2, 0x02, 0x01, // FEATURE (Data,Var,Abs,Buf)
0xc0 // END_COLLECTION
};

and the actual data array is:

extern uint32_t time;
extern uint32_t counter;
extern uint32_t temp_time;
extern uint32_t temp_counter;

extern uint8_t small_counter;

uchar usbFunctionRead(uchar *data, uchar len)
{
if(len > bytesRemaining) len = bytesRemaining;
/*eeprom_read_block(data, (uchar *)0 + currentAddress, len);
currentAddress += len;
bytesRemaining -= len;*/
//int16_t time0 = get_timer(); // not precise in milisecunds should be 11.71875
small_counter = TCNT0;

data[0] = *(((uchar*) &time) + 0);
data[1] = *(((uchar*) &time) + 1);
data[2] = *(((uchar*) &time) + 2);
data[3] = *(((uchar*) &time) + 3);
data[4] = *(((uchar*) &small_counter) + 0);
data[5] = *(((uchar*) &counter) + 1);
data[6] = *(((uchar*) &counter) + 2);
data[7] = *(((uchar*) &counter) + 3);

So you are using 8 chars and sending 128 Bytes over USB. You ignore the rest, but your buffer must be 128 bytes long or your app crashes. Not sure if there is a growing offset between the microcontroller time basis and PC.


I am still waiting for the sources of the v3 detectors to be released.

Profile TJM
Project administrator
Project developer
Project tester
Send message
Joined: 16 Apr 11
Posts: 291
Credit: 1,103,478
RAC: 372

Message 2388 - Posted: 28 Feb 2014 | 9:55:17 UTC - in response to Message 2368.

In theory, the BOINC app can run standalone. Without any input and config it will run and log data for 1,5 hour. With a simple .bat file it is possible to log data continuously for extended periods of time. I know it's not an ideal solution, but the app was never really designed (and tested) for this mode of operation, as it's primary purpose is to run under BOINC control.

phys
Avatar
Send message
Joined: 28 Apr 12
Posts: 24
Credit: 0
RAC: 0
Message 2389 - Posted: 1 Mar 2014 | 2:19:18 UTC

I have put a script together, to access the device in your favourite operating system (Win, Linux, Arm,..) using the cross platform library hidapi and python. You can get this python module on http://https://github.com/trezor/cython-hidapi/ and compile it yourself. For your convenience and windows users I have put it together with the script in the zip file. Put the hid.pyd module into C:\Python27\Lib\site-packages.(Please use the IDLE python GUI in windows, or you might get an error on the MS-CMD-Prompt)

A sample output looks like


==============================
Radiactive At Home Test Script
==============================

Show you all the HID devices on your PC
interface_number : -1
manufacturer_string : microchip
path : \\?\hid#vid_04d8&pid_f6fe#7&8095337&0&0000#{4d1e55b2-f16f-11cf-88cb-001111000030}
product_id : 63230
product_string : radioactiveathome.org GRS
release_number : 769
serial_number : Љ
usage : 1
usage_page : 65280
vendor_id : 1240

Opening selected device
Manufacturer: microchip
Product: radioactiveathome.org GRS
[0, 184, 160, 46, 0, 28, 135, 0]
Realtime clock: 3055.8s
Counts: 34588
Realtime clock: 3065.8s, Counts: 34705
Realtime clock: 3075.8s, Counts: 34823
Realtime clock: 3085.81s, Counts: 34927
Realtime clock: 3095.81s, Counts: 35055
679.402 CPM, 3.96847 mikroSv/h
Corrected 680.944 CPM, 3.97748+-0.0211957 mikroSv/h
Closing device
Done


You can get the package here and extend it to your own needs.

Happy hacking!

phys
Avatar
Send message
Joined: 28 Apr 12
Posts: 24
Credit: 0
RAC: 0
Message 2390 - Posted: 1 Mar 2014 | 12:43:31 UTC

As I can't edit the post, here is the updated link

Profile jhelebrant
Avatar
Send message
Joined: 30 Jul 12
Posts: 27
Credit: 1,521
RAC: 0
Message 2551 - Posted: 16 Apr 2014 | 13:05:03 UTC - in response to Message 2390.

Hi,
thank you very much for your help. My colleague modified the script to write the data to text file + some improvements so it can be for example adjusted for various time intervals etc. You can get it here from his github:
https://github.com/bulanek/rad/blob/master/rad.py

gismoat
Send message
Joined: 5 Apr 13
Posts: 27
Credit: 146,764
RAC: 0

Message 2970 - Posted: 13 Dec 2014 | 23:46:51 UTC

Hello,

quick question, how could i use this script? Does he need a plugin for Boinc?

thanks
Werner

phys
Avatar
Send message
Joined: 28 Apr 12
Posts: 24
Credit: 0
RAC: 0
Message 3014 - Posted: 24 Jan 2015 | 16:00:46 UTC - in response to Message 2970.

Follow the given links. Wikipedia explains Phython programmming language pretty well.

--Norbert
Send message
Joined: 6 Jan 15
Posts: 69
Credit: 0
RAC: 0
Message 3016 - Posted: 25 Jan 2015 | 5:07:56 UTC - in response to Message 3014.

For the above scripts to work, here is some additional help that first-time Windows - Python users will need (i.e. problems I just had):

When you install Python for Windows, it must be version 2.7, not version 3.4.

When you choose the Windows installer for version 2.7, it must be the "Windows x86 MSI installer", not the "Windows x86-64 MSI installer", even if you have 64 bit Windows. The 32 bit version is required for the hid.pyd module to work from C:\Python27\Lib\site-packages.

The original test script published by phys works as-is (you may need to tell your browser to ignore certificate problems with the owncloud site that the link points to).

The modified version of the script provided by jhelebrant needs a change to work on a Windows machine. Line 37, OUTPUT_PATH="/tmp" must be changed to OUTPUT PATH="C:\Python27\MyFiles" [or similar; make sure the folder exists].

--Norbert

--Norbert
Send message
Joined: 6 Jan 15
Posts: 69
Credit: 0
RAC: 0
Message 3069 - Posted: 17 Feb 2015 | 5:59:41 UTC
Last modified: 17 Feb 2015 | 5:59:51 UTC

Now that I have a working detector (no display yet, but the LED flashes, and the sound/LED button works), I can't seem to use the script. The script lists all the USB devices properly (but no radioactiveathome.org string), and finishes with

Opening selected device
open failed
You probably don't have the hard coded hid. Update the hid.device line
in this script with one from the enumeration list output above and try again.


What is not clear is *How* to update the hid.device line. This line in the script shows as follows:
print("Opening selected device")
h = hid.device()
, and presumably the "hard coded HID" goes between the brackets. But what is that? There is no field of this name. The string that includes hid# does not work in any permutation, for any of the devices in my list. The other fields don't work either.

And how do we know that the detector is properly enumerated? If there is no 'product_string' containing "radioactiveathome" then does this indicate a USB problem? Does Windows need a driver to recognize the detector?

--Norbert

phys
Avatar
Send message
Joined: 28 Apr 12
Posts: 24
Credit: 0
RAC: 0
Message 3231 - Posted: 16 May 2015 | 17:01:54 UTC - in response to Message 3069.

Hi Norbert,

There are so many different sensors going around and no one lists the differences between them, my requests for details were not answered. I've got the 3rd Version Sensor, the tube shaped one with PIC micro-controller, that is internally named as v2.51 sensor.

Go to Windows Device Manager and look under Input Devices (Human Interface Devices). There should be a HID-compliant device. Click on it and go to details tab. Now choose Hardware ID. Mine is HID\VID_04D8&PID_F6FE&REV_0301.

Next open up the Python Script. Look for

try:
print("Opening selected device")
h = hid.device()
h.open(0x4d8, 0xf6fe)


Change it to your PID and VID.

phys
Avatar
Send message
Joined: 28 Apr 12
Posts: 24
Credit: 0
RAC: 0
Message 3232 - Posted: 16 May 2015 | 18:12:58 UTC

Now a had a look on the source code of the 2.61 Kit, as a know where to look for.(usbconfig.h)


#define USB_CFG_VENDOR_ID 0xc0, 0x16 /* = 0x16c0 */
/* USB vendor ID for the device, low byte first. If you have registered your
* own Vendor ID, define it here. Otherwise you may use one of obdev's free
* shared VID/PID pairs. Be sure to read USB-IDs-for-free.txt for rules!
* *** IMPORTANT NOTE ***
* This template uses obdev's shared VID/PID pair for Vendor Class devices
* with libusb: 0x16c0/0x5dc. Use this VID/PID pair ONLY if you understand
* the implications!
*/
#define USB_CFG_DEVICE_ID 0xdf, 0x05 /* = 0x05df */
/* This is the ID of the product, low byte first. It is interpreted in the
* scope of the vendor ID. If you have registered your own VID with usb.org
* or if you have licensed a PID from somebody else, define it here. Otherwise
* you may use one of obdev's free shared VID/PID pairs. See the file
* USB-IDs-for-free.txt for details!
* *** IMPORTANT NOTE ***
* This template uses obdev's shared VID/PID pair for Vendor Class devices
* with libusb: 0x16c0/0x5dc. Use this VID/PID pair ONLY if you understand
* the implications!
*/
#define USB_CFG_DEVICE_VERSION 0x61, 0x02
/* Version number of the device: Minor number first, then major number. */
//#define USB_CFG_VENDOR_NAME 'o','b','d','e','v','.','a','t' //8
#define USB_CFG_VENDOR_NAME 'r','a','d','i','o','a','c','t','i','v','e','a','t','h','o','m','e','.','o','r','g' //21
#define USB_CFG_VENDOR_NAME_LEN 21
/* These two values define the vendor name returned by the USB device. The name
* must be given as a list of characters under single quotes. The characters
* are interpreted as Unicode (UTF-16) entities.
* If you don't want a vendor name string, undefine these macros.
* ALWAYS define a vendor name containing your Internet domain name if you use
* obdev's free shared VID/PID pair. See the file USB-IDs-for-free.txt for
* details.
*/
//#define USB_CFG_DEVICE_NAME 'r','a','d','i','o','a','c','t','i','v','e','a','t','h','o','m','e','.','o','r','g',' ','G','R','S' //25 signs
#define USB_CFG_DEVICE_NAME 'R','a','d','i','a','t','i','o','n',' ','D','e','t','e','c','t','o','r',' ','K','I','T' //22
#define USB_CFG_DEVICE_NAME_LEN 22
/* Same as above for the device name. If you don't want a device name, undefine
* the macros. See the file USB-IDs-for-free.txt before you assign a name if
* you use a shared VID/PID.
*/
/*#define USB_CFG_SERIAL_NUMBER 'N', 'o', 'n', 'e' */


So the team (ignored all warnings) and used VID=16c0 with a new PID=05df.

Note to myself: In the published source code of the v3.01 Sensor the VID and PID was changed to XXXX. If I ever want to compile it on my own or change anything, I have to insert PID and VID again because they were deleted.

Post to thread

Message boards : Science : Data logging in offline mode


Main page · Your account · Message boards


Copyright © 2019 BOINC@Poland | Open Science for the future