[NXTasy][NXC] NXC Profiler

[muntoo]

This is Google's cache of http://forums.nxtasy.org/index.php?show ... ded&start=. It is a snapshot of the page as it appeared on 24 Aug 2010 02:02:37 GMT. The current page could have changed in the meantime. Learn more

Text-only version
These search terms are highlighted: nxc profiler nxtasy

HomePortalHelpSearchMembersCalendar
Welcome Guest ( Log In | Register )
nxtasy.org forums > LEGO Mindstorms NXT > NXT Software
Project proposal: NXC Profiler, Such a tool would be great for debugging
Options
Would you use or even develop a NXC profiler?
Do you consider this idea useful? Would you use the tool?
No, I don't think so. [ 2 ] [13.33%]
I can't say, maybe I'd give it a try. [ 9 ] [60.00%]
Yes, I'd love to use it. [ 4 ] [26.67%]
Would you be interested in developing a profiler?
No, I couldn't do that. [ 5 ] [33.33%]
No, I don't think that's possible. [ 2 ] [13.33%]
Yes, but it might be too hard for me / I'd need help. [ 6 ] [40.00%]
Yes, and I think I could do that (not necessarily alone). [ 2 ] [13.33%]
Total Votes: 15
Guests cannot vote
Linus
Nov 29 2009, 07:10 PM
Post #1


Loving to code


Group: Members
Posts: 960
Joined: 21-March 08
From: Aachen, Germany
Member No.: 4,951



In this thread I'll try to explain an idea which I have, but which I won't be able to work on for the next couple of days / weeks. So let's hope somebody likes this project idea and maybe starts development on their own. Let's give it a shot:

A profiler is a tool that measures performance of your programs. Usually you start the program with the profiler enabled, and as a result you get the sourcecode in a special view: The lines which take longest to execute are highligted or colored in some way. Also you usually see how many times each line of code was called, and how much execution time was spent there.

This is very very helpful when you want to optimize your code for performance. It's easy to spot bottlenecks. It's a great way to try several ways to do certain things and to compare runtimes (which methods work best).

Apart from the benefit for performance analysis and optimization, it could also be a great way to debug: You could easily spot dead code, i.e. parts of your program that are never executed. Also it might be helpful to detect where a program crashes, if the line number is indicated where execution stopped (I know this is partially a different problem).


So my idea would be to develop a sort of "NXC preprocessor". It would take syntactically correct NXC-files and "add stuff to them" to be executed at runtime. This would mean you'd create a totally new RXE file from a preprocessed / generated NXC file. This file would probably execute a lot slower, and it would probably be considerably bigger. Also there would be the unlikely (?) case of non-profilable programs, which are either huge or require a lot of free RAM.


How would this work? I'd try to include macros / lines of executable code between every line of original NXC sourcecode. If that is too much, we could start with every other line or every N-th line or with a line at the start of every function / every task. This "additional profiling instructions" would just increase the "I've been here" counter and somehow record the time.

I know we only have a timing accuracy of 1ms, this could pose a problem. But their might be tricks and workarounds (either technical or statistical), or we could just do the timing thing for bigger code-blocks or whole functions.

The question is what to do with the collected data after the program has finished. It could of course be written on the NXT's flash memory to be read out and analyzed by software on a PC. Or it could be directly transmitted via mail-messages and direct commands. The profiler could then interpret these results and generate a HTML (or whatever else) version of the original NXC sourcode...


A big question is of course my second sub-problem: What if the program crashes? Well, the temporary profiling results would only be availabe if they were saved during execution. Saved on flash that is. So this would be very very slow.
We could simplify this case to a way of storing only the last line number. The inserted profiler-code would just store the line number it was in last. We'd only need a single "globally accessable" variable or register that survives a crashing program. How'd we do that? I see a possibility with John Hansen's enhanced firmware (somewhere ought to be 4 free bytes in RAM I hope?). Or one could take advantage of unused sensor/motor IO map registers for this. Very easy solution: Leave 1 unused motor port and take the TachoLimit (easily settable and readable) as line-numbers. It is questionable if this really survives an NXC program crash...

The profiling data / statistics could of course also be collected via direct command messaging, but I believe this method would be too slow and too inaccurate.


So what do you think? Am I crazy? Is this unrealistic? Would it help you or not? I think it's definitely valuable if you make heavy use of algorithms in your programs or if you push your NXT to the limits...




Replies
Linus
Dec 12 2009, 04:00 PM
Post #2


Loving to code


Group: Members
Posts: 960
Joined: 21-March 08
From: Aachen, Germany
Member No.: 4,951



Spiller,
thank you for the idea. Indeed, I wasn't happy with the top tables. But I decided to keep them and transformed them to real tables now (the <pre> and <code> thing with the &nbsp;s was just a stupid way of manually doing a table-like thingy).

I also did a major update again, with better and more accurate statistics. I also added another example. There you can see that in my sample run, the Ultrasonic sensor never encountered a target. Also you easily spot that calling SensorUS() takes 30ms (no big news, I know).

http://www.mindstorms.rwth-aachen.de/tr ... XCProfiler


By now I think it's "usable", so if you people want to give it a try, go ahead. Nested PROFILER_BEGINSECTION commands are not allowed. I'd keep the order of the section-numbers "correct in order". Order them by line-numbers (not by functions or whatever), so that PROFILER_BEGINSECTION(0) is the section in the top of the program (smallest line number). No big deal though.

Also there is NO support for #included statements. You can use your program of course, but the Analyzer will only analyze the main file (or an included file, if you say so). But only 1 nxc-file.




Posts in this topic
Linus Project proposal: NXC Profiler Nov 29 2009, 07:10 PM
Spiller When optimizing programs this could be very useful... Nov 29 2009, 08:07 PM
Linus When optimizing programs this could be very useful... Nov 29 2009, 08:33 PM
Xander Soldaat Linus, Although my NXC programming skills are not... Nov 30 2009, 02:15 AM
Linus I thought about this a little more, and here's... Dec 1 2009, 02:19 PM
John Hansen For the record, I voted "No" and "N... Dec 1 2009, 03:47 PM
Linus For the record, I voted "No" and "N... Dec 1 2009, 04:32 PM
John Hansen With the enhanced NBC/NXC firmware you can do some... Dec 1 2009, 04:49 PM
Linus With the enhanced NBC/NXC firmware you can do some... Dec 1 2009, 05:08 PM
John Hansen Is there a tutorial or document about this step-by... Dec 1 2009, 07:08 PM
Xander Soldaat This sounds like a great job for Xander Soldaat... Dec 2 2009, 02:06 AM
Linus I couldn't resist to start on a little proof-o... Dec 7 2009, 08:09 PM
Linus Small update: First part of the Python-Analyzer is... Dec 7 2009, 10:56 PM
Linus I'm proud to announce the first "working ... Dec 11 2009, 12:07 AM
Linus It's me again, with yet another update. But th... Dec 11 2009, 10:38 AM
Spiller Looking nice, I will try it out on the NXT later w... Dec 11 2009, 11:44 AM
Linus Spiller, thank you for the idea. Indeed, I wasn... Dec 12 2009, 04:00 PM
Linus Btw, I'd still be very happy if someone commen... Jan 7 2010, 07:37 PM
snipeytje your link is broken. you have to remove the ) from... Jan 8 2010, 02:46 AM
Linus and how can i download it because when i select th... Jan 8 2010, 03:18 AM
snipeytje found it thanks for the explanation Jan 8 2010, 10:11 AM
mike1 I'm not sure if this would work, but what abou... Jan 8 2010, 10:40 AM
snipeytje why dont you publish it as an exe instead of a pyt... Jan 9 2010, 09:39 AM
Linus why dont you publish it as an exe instead of a pyt... Jan 9 2010, 10:05 AM
snipeytje cant u make a UI? Jan 9 2010, 10:36 AM
Linus cant u make a UI? Yes I could, but I was hoping ot... Jan 12 2010, 06:10 AM
Spiller Interestingly, I tried it for a multi-file project... Jan 12 2010, 06:58 AM
Linus I thought the limitation was in the .nxc file and ... Jan 12 2010, 08:51 AM
Spiller I haven't really had the chance to try it out;... Jan 11 2010, 06:26 PM
snipeytje Yes I could, but I was hoping others maybe could a... Jan 12 2010, 11:26 AM
Linus i might try making a simple ui for the script that... Jan 12 2010, 11:42 AM
Spiller I have tried with other applications where it was ... Jan 12 2010, 12:44 PM
Linus (Is the .prf file always called "sample... Jan 12 2010, 01:11 PM
Spiller No, actually it's what you call it in this lin... Jan 13 2010, 01:40 PM
snipeytje my pc also would accept the python command but i u... Jan 14 2010, 12:49 AM

« Next Oldest · NXT Software · Next Newest »







Lo-Fi Version Time is now: 23rd August 2010 - 09:02 PM
Powered By IP.Board © 2010 IPS, Inc.