logging data sometimes is blank

stingg · Post by **stingg** » Fri Jan 03, 2014 8:59 am

This blank data is creating alot of issue when converting it to another file format.

Post by **brentp** » Sat Jan 04, 2014 3:54 am

Hi,

Can you elaborate on this some more? The log file is specifically designed to log channels at different rates; the slower channels have blank gaps where there is no update. If you load it into a spreadsheet, you will see which channels update faster than others.

What file format are you trying to convert to?

GTIspirit · Post by **GTIspirit** » Fri Jan 10, 2014 2:59 am

stingg wrote:This blank data is creating alot of issue when converting it to another file format.

This has also affected me, but with some creativity there are some solutions. A data analysis program I've used to look at RCPro data needs all signals recorded at the same rate to be in their own file with its own time axis and no spaces between rows. So 10Hz signals would go in one file with time points every 100ms, 1Hz in a separate file with time points every 1s, etc.

My solution was to import the data to Excel, group the columns by measure rate, and then use the Excel filter feature to filter out the blanks, leaving only the rows with values, along side the time axis. This can then be copied and pasted and during the copy paste action Excel will leave out the blank rows if the filter is active.

Very occasionally there are inconsistencies in the data that require a manual touch, but those are easy enough to fine with clever manipulation of the filter.

Post by **brentp** » Fri Jan 10, 2014 3:05 am

The sparse logging we use to allow flexible logging rates per channel is important for efficient operation of the firmware. However, in Race Analyzer we could create an export feature that duplicates all values. Would that be helpful?

GTIspirit · Post by **GTIspirit** » Sat Jan 11, 2014 3:25 am

Thanks for offering, usefulness depends on the implementation. And what's useful to me might not be useful for others.

What would be particularly helpful is a proper time stamp/recording time column. When I post-process the RCPro data in Excel I make up a time column based on the assumptions all samples were acquired at exactly the interval of the fastest logging rate. e.g. if rpm is acquired at 100Hz, every row has a rpm entry, so I assume they are exactly every 10ms. Then I line up channels acquired at different rates with that artificially created recording time.

I'm guessing this would help a lot of people who choose an external data processor and want to plot a signal versus time and end up having to follow a similar procedure as me to make up the recording time axis.

What I wouldn't want to see is filler values between samples. For example, if I log engine temp at 1Hz, and engine speed at 100Hz, I wouldn't want to see 99 filler values for engine temp between actual sample values.

For me it would be super helpful to export the signals based on acquisition rate. All signals recorded at 100Hz in one file, all recorded at 1Hz in another file, etc.

Post by **brentp** » Sun Jan 12, 2014 5:04 am

Righto. Sounds like the solution for you is to write a python or ruby script to translate the logfile to the format you specifically need!

jpf11 · Post by **jpf11** » Mon Jan 13, 2014 10:59 pm

I too had the issue of empty data fields (amongst other issues like invalid data and unit incompatibilities) when trying to convert the rcpro csv output into something usable in GEMS and AEM DA. You can see what we did in this thread:

http://www.autosportlabs.org/viewtopic.php?t=3731

You can find my perl script that I wrote there for tackling the various issues.

Autosport Labs

logging data sometimes is blank

logging data sometimes is blank

Re: logging data sometimes is blank