Data Analysis Software - Motec, Gems, AEM
Moderators: JeffC, rdoherty, stieg, brentp
Hey folks, I need help on data analysis.
I've been look at the posts made here and I don't understand half of what you guys are talking about.
Can someone help me with a step-by-step? What software do I need? I have the GEMS Data Analysis 4.1.1. I have ActivePerl 5.20.1.2000.
Right now, I only have logs with the car standing in the garage to do test.
I get .log files on the SD card.
What do I do from there. I'm not a programmer, so Perl is uncharted for me. I've checked the GitHUB, but the readme file is not really helping
Please help me. The first race for my team is coming up in 2 weeks and my co-drivers are eager to look at the data.
I've been look at the posts made here and I don't understand half of what you guys are talking about.
Can someone help me with a step-by-step? What software do I need? I have the GEMS Data Analysis 4.1.1. I have ActivePerl 5.20.1.2000.
Right now, I only have logs with the car standing in the garage to do test.
I get .log files on the SD card.
What do I do from there. I'm not a programmer, so Perl is uncharted for me. I've checked the GitHUB, but the readme file is not really helping
Please help me. The first race for my team is coming up in 2 weeks and my co-drivers are eager to look at the data.
-
- Posts: 30
- Joined: Sun Mar 15, 2015 3:03 am
- Location: United States
I'm using this for perl. http://padre.perlide.org/download.html
1st, install that. What you mentioned will probably work, but I'm unfamiliar with it, so no idea for sure.
2nd, go find Dlog99 on the GEMS site, you'll need that
3rd, put the script referenced earlier into a notepad file named rccsv2gems.pl, put it in a folder containing the datalog files, ideally not on the SD card.
4th, open up the command window. In Windows 7 just type "cmd" in the search bar, older versions you need to go run->"cmd".
5th, in the command window you need to navigate to the folder you created. Type "cd *path*" where *path* is how you get to the folder you created in step 3, something like "C:\Users\copper280z\Desktop\Data", don't include quotes or asterisks
6th, in the command window type "perl rccv2gems.pl *input_filename* *output_filename* --disable-gps-cleanup", where *input_filename* and *output_filename* are the file names you want to use EXACTLY as they appear in the folder, INCLUDING the extensions, don't include the *'s. The output file won't exist yet, but will be created when you execute the script, I suggest using the *.csv extension. I chose to have you disable the gps satellite cleanup since you probably didn't have any while sitting in your garage, it'll make sure you have the whole time you recorded.
7th, fire up dlog99 and go file-> import, follow the prompts and save it as a *.stf file. Make sure to select floating point instead of the default integer on the import window.
8th, open up that *.stf file in GEMS Data Analysis! (Or AEMData)
1st, install that. What you mentioned will probably work, but I'm unfamiliar with it, so no idea for sure.
2nd, go find Dlog99 on the GEMS site, you'll need that
3rd, put the script referenced earlier into a notepad file named rccsv2gems.pl, put it in a folder containing the datalog files, ideally not on the SD card.
4th, open up the command window. In Windows 7 just type "cmd" in the search bar, older versions you need to go run->"cmd".
5th, in the command window you need to navigate to the folder you created. Type "cd *path*" where *path* is how you get to the folder you created in step 3, something like "C:\Users\copper280z\Desktop\Data", don't include quotes or asterisks
6th, in the command window type "perl rccv2gems.pl *input_filename* *output_filename* --disable-gps-cleanup", where *input_filename* and *output_filename* are the file names you want to use EXACTLY as they appear in the folder, INCLUDING the extensions, don't include the *'s. The output file won't exist yet, but will be created when you execute the script, I suggest using the *.csv extension. I chose to have you disable the gps satellite cleanup since you probably didn't have any while sitting in your garage, it'll make sure you have the whole time you recorded.
7th, fire up dlog99 and go file-> import, follow the prompts and save it as a *.stf file. Make sure to select floating point instead of the default integer on the import window.
8th, open up that *.stf file in GEMS Data Analysis! (Or AEMData)
Last edited by Copper280z on Fri Apr 03, 2015 5:23 am, edited 1 time in total.
Hi,
Just updated the script here with the latest from this thread, updated the documentation and included a sample log file:
https://github.com/autosportlabs/RCP2GEMS
Script seems to work with the sample log file included.
@PopKorn78, you can download the file directly from the site above- after navigating to the site, click the "Download Zip" option on the right side.
Thanks for all the help!
Just updated the script here with the latest from this thread, updated the documentation and included a sample log file:
https://github.com/autosportlabs/RCP2GEMS
Script seems to work with the sample log file included.
@PopKorn78, you can download the file directly from the site above- after navigating to the site, click the "Download Zip" option on the right side.
Thanks for all the help!
-
- Posts: 30
- Joined: Sun Mar 15, 2015 3:03 am
- Location: United States
After running the test file through the script(with gps cleanup disabled), I'm seeing something weird happen at row 13933, time jumps from 266sec to 2.76e8. It happens in your example log. I also cannot import the parsed log into dlog99, it throws an invalid floating point operation error, probably because of the time.brentp wrote:Hi,
Just updated the script here with the latest from this thread, updated the documentation and included a sample log file:
https://github.com/autosportlabs/RCP2GEMS
Script seems to work with the sample log file included.
@PopKorn78, you can download the file directly from the site above- after navigating to the site, click the "Download Zip" option on the right side.
Thanks for all the help!
-
- Posts: 70
- Joined: Tue Mar 10, 2015 3:31 pm
- Location: Detroit, MI
Having troubles with the latest version of the script (1.7). Any help would be appreciated.
First off, I'm using data collected from the car while sitting in the garage, so all 0's for the GPS data channels.
When I run with GPS cleanup enabled I get stuck in the while loop that outputs the following message:
Then I tried disabling GPS cleanup. Here's a copy from the command prompt of what happens:
So it runs out of memory, again - I'm guessing that it's the while loop starting @ line 376.
File is only 1868 rows long.
The demo file that is ~114,000 rows long runs without issue.
Log file that gives the perl script issues is attached.
Thanks in advance!
First off, I'm using data collected from the car while sitting in the garage, so all 0's for the GPS data channels.
When I run with GPS cleanup enabled I get stuck in the while loop that outputs the following message:
It sits and runs that loop (while loop starting @ line 293) until I stop the program."line $i, latitude is \"$array[$i][$lat_col]\", longitude is \"$array[$i][$long_col]\", erroneous values, tagging line for removal\n"
Then I tried disabling GPS cleanup. Here's a copy from the command prompt of what happens:
C:\Windows\System32>cd C:\Users\JoshMcD\Documents\RaceCapturePro\DataFiles\RCP2G
EMS-master_v1.7\RCP2GEMS-master
C:\Users\JoshMcD\Documents\RaceCapturePro\DataFiles\RCP2GEMS-master_v1.7\RCP2GEM
S-master>perl rccsv2gems_debug.pl RC_17.log RC_17_test1
rccsv2gems.pl - Race Capture Pro CSV to GEMS DA conversion utility v1.0
Filtering data for a minimum of 4 GPS satellites.
Disabling clean-up of GPS (if your data has bad gps data it's up to you to figur
e out how to get your track to draw properly in AEM/GEMS Data Analysis).
Initiating conversion...
Initiating Header Cleanup - Cleans up column headers.
Header Cleanup Completed Successfully!
Initiating Moving Time Column to Front - Moves the Time column to the first colu
mn.
Time Column to Front Completed Successfully!
Initiating Junk Line Removal Tagging - looking for lines with non-numerical data
and tagging for removal during file output.
Junk Line Removal Tagging Completed Successfully!
Initiating Pre-GPS initialization - Finds the first real gps lat/long values the
n sets all prior lat/long to match.
Out of memory!
C:\Users\JoshMcD\Documents\RaceCapturePro\DataFiles\RCP2GEMS-master_v1.7\RCP2GEM
S-master>
So it runs out of memory, again - I'm guessing that it's the while loop starting @ line 376.
File is only 1868 rows long.
The demo file that is ~114,000 rows long runs without issue.
Log file that gives the perl script issues is attached.
Thanks in advance!
- Attachments
-
- RC_17.LOG
- (146.32 KiB) Downloaded 1594 times
Josh
-
- Posts: 30
- Joined: Sun Mar 15, 2015 3:03 am
- Location: United States
Look at my repo, I made an edit for you. This was based on the copy I had locally, not Brent's new one, so there may be differences. I also changed a bunch of indentation a while ago, so that's showing up in the changes too.
https://github.com/Copper280z/RCP2GEMS/ ... 376d9c1192
Looks like $#array was increasing as the counter increased, I'm not versed in perl well enough to understand why, so I just made a new variable that holds the starting length, that seemed to do it.
Your data also has the Utc column as zeros, not sure why. The script doesn't find anything non-zero so it exits. Brent, is that column derived from GPS?
I didn't do it in my commit, but if you want to look at this data you can edit line 139 from this:
to this:
https://github.com/Copper280z/RCP2GEMS/ ... 376d9c1192
Looks like $#array was increasing as the counter increased, I'm not versed in perl well enough to understand why, so I just made a new variable that holds the starting length, that seemed to do it.
Your data also has the Utc column as zeros, not sure why. The script doesn't find anything non-zero so it exits. Brent, is that column derived from GPS?
I didn't do it in my commit, but if you want to look at this data you can edit line 139 from this:
Code: Select all
if ( $array[0][$i] =~ /Utc/ ) {
Code: Select all
if ( $array[0][$i] =~ /Interval/ ) {
-
- Posts: 70
- Joined: Tue Mar 10, 2015 3:31 pm
- Location: Detroit, MI
I did some logs where the GPS had signal. Script ran flawlessly. For the purposes of garage/indoor testing (no GPS data or GPS wire disconnected) it would still be nice to have the script see that there's no GPS data and use Interval insteadbrentp wrote:The UTC column is derived from GPS, correct.
Could you try a log from a drive around the block, with valid GPS data?
Thanks for following up!
Josh
Josh
-
- Posts: 4
- Joined: Mon Jan 07, 2013 8:54 pm
So, in version 4.1.1 build 920, the import csv function appears to be dead. I can only get it when in project view and close the project. There are three options, open log file, import csv, download log file. The other options create new dialogs but the import csv does not appear to do anything. Anyone able to validate this? What versions are being run successfully and do you have the install for it?
-
- Posts: 4
- Joined: Mon Jan 07, 2013 8:54 pm
We had some trouble with perl so went python and had to make some changes today on github.
Steps I took after using the perl/python script.
1. You must get the Dlog99 package at the bottom of the page here (not the current analysis package!):
http://www.gems.co.uk/?content=pages&id ... -downloads
2. You must import the csv in the Dlog99 software, choosing float for the numbers.
3. Save the stf file.
4. Now you can open this stf file with the GDA - Gems Data Analysis software
5. You must go to the little wrench in each field and choose the actual channel names from the RCP. You must also change the track map source to GPS.
So far so good. Need to get a handle on their file management, projects, etc to make this workable in our team environment. Saving these variables so I don't have to set the windows every time will be big.
Result:
https://www.dropbox.com/s/f8yg9i2u5s4qi ... 3.mp4?dl=0
Steps I took after using the perl/python script.
1. You must get the Dlog99 package at the bottom of the page here (not the current analysis package!):
http://www.gems.co.uk/?content=pages&id ... -downloads
2. You must import the csv in the Dlog99 software, choosing float for the numbers.
3. Save the stf file.
4. Now you can open this stf file with the GDA - Gems Data Analysis software
5. You must go to the little wrench in each field and choose the actual channel names from the RCP. You must also change the track map source to GPS.
So far so good. Need to get a handle on their file management, projects, etc to make this workable in our team environment. Saving these variables so I don't have to set the windows every time will be big.
Result:
https://www.dropbox.com/s/f8yg9i2u5s4qi ... 3.mp4?dl=0
-
- Posts: 2
- Joined: Fri May 29, 2015 7:44 am
- Location: Melbourne, Australia
Hey guys,
Having some issues with GEMS itself.
When you have the data loaded and try to switch from time to distance mode it's goes nuts.
As viewing data by time is nearly useless for comparison.
Any help appreciated,
Mat Walker
DREAM Motorsport (F4, FFord)
Pedders Racing ( GT86 Australian Production Touring Championship)
Having some issues with GEMS itself.
When you have the data loaded and try to switch from time to distance mode it's goes nuts.
As viewing data by time is nearly useless for comparison.
Any help appreciated,
Mat Walker
DREAM Motorsport (F4, FFord)
Pedders Racing ( GT86 Australian Production Touring Championship)
Mat Walker
DREAM Motorsport (F4, FFord, Aussie Sports Sedans)
Pedders Racing ( GT86 Australian Production Touring Championship)
Racing cars upside down adds an extra dimension to the problem
- Down under motorsport problems
DREAM Motorsport (F4, FFord, Aussie Sports Sedans)
Pedders Racing ( GT86 Australian Production Touring Championship)
Racing cars upside down adds an extra dimension to the problem
- Down under motorsport problems