Mindboards

Nicolas Schodet wrote: Hi,

I am working on compiling the firmware with GCC in order to make it easy for anybody to hack the original firmware.

I would like to make it easier for any curious user to work on the firmware using GCC or IAR, to share its work, and to track changes made to the original firmware. For this, I am building a site which will contain instructions on how to build the firmware, which will try to explain firmware internals, and which provides a decentralised source repository.

I started this project after some frustration about:

• NXTGCC:
  • not stable enough,
    distributed with a full Windows toolchain and Eclipse installation for which I have no usage,
    no source repository, poor visibility on what changed and why,
    no IAR support for those who like it.
  NXC Enhanced firmware:
  • can not compile using GCC,
    no source repository, bad visibility on what changed and why,
    no release versions, so its hard to know what is exactly in a binary.
    other firmware:
    not compatible with NXC which I like to use.

I greatly appreciate work by John Hansen and Rasmus Ulslev Pedersen, I just want more


I am going on vacation Saturday until 2010-08-08, so this website will not be ready before I leave, but you can try some firmware yet. I think I have solved problems of NXTGCC (you will find how in the related commits).

So if you want to try it, download http://ni.fr.eu.org/~nico/nxt_firmware.rfw to your brick and tell me about any problem you can see. Warning: this will erase all the files in your user file system. This is a stock 1.29 firmware.

I you would like to test GCC compilation before the website is ready, please let me known.

Ford Prefect wrote: hi,
sounds cool!
can you please tell what's it all about with this fw?
is there a home page we can read sth about your fw development?
what's different?
what's better?
what's easier?
which IDE to use?
what syntax - c or c++ or NXC?
support of dynamic memory, pointers, recursion?
what API?
do you have program examples maybe using multithreading (bumper car, line follower, maze solver)?
informations greatly appreciated!

Nicolas Schodet wrote:

Ford Prefect wrote: hi,
sounds cool!
can you please tell what's it all about with this fw?
is there a home page we can read sth about your fw development?

Not yet, but I am working on it. It should be ready in three weeks.

Ford Prefect wrote: what's different?
what's better?
what's easier?
which IDE to use?
what syntax - c or c++ or NXC?
support of dynamic memory, pointers, recursion?
what API?
do you have program examples maybe using multithreading (bumper car, line follower, maze solver)?
informations greatly appreciated!

For the moment, it should be exactly the same as the original LEGO firmware. The goal is to permit people to make change easily to the original firmware. There is no benefit yet for the end user. There is only benefit for people wanting to experiment with modifying the firmware.

I am leaving now, see you in two weeks

John Hansen wrote: Some of my "enhancements" involve refactoring and changes to reduce the code and data size of the resulting build. Are you willing to have those kind of check-ins or would you prefer to limit changes to specific feature additions and bug fixes?

One change, for example, involved making the compiler avoid as much as possible all float to double or vice versa conversions that were showing up in the IAR map file. So floating point constants were declared with a trailing F or prepended with (float) in a few places. Where the 1.28 source code had calls to sin, cos, tan, etc... (removed from the compiled code via #if 0) I used the sinf, cosf, tanf, etc... functions instead. Another example is that I removed duplicate strings from the UI.txt file and rearranged the constants used in Functions.inl which referred to those duplicate strings. Repetitive code in c_input.c within a large case statement has been extensively refactored as well. None of these are necessarily enhancements or bug fixes and it makes it harder for me to do comparisons back to the original LEGO code but I would prefer to convince LEGO to switch to my implementation rather than consider rolling back those kind of changes in my code.

John Hansen

Ford Prefect wrote: ... but please no reduction for double to float but enhancement for double instead if possible )

TC Wan wrote:

Nicolas Schodet wrote:
No you really need an EABI toolchain.

This is explained here: http://nxt-firmware.ni.fr.eu.org/build/gcc/

Hmm. I'm still confused regarding the EABI vs. ELF toolchains. Googling doesn't reveal much info.
Prior to this I was building NxOS successfully using the Yagarto ELF toolchain on Mac OSX, and it seems to run fine.
According to ARM Info Center, they've adopted EABI since ~2003 and the BSABI document (ARM IHI 0036B, current through ABI release 2.08 10th October 2008, reissued 28th October 2009) does not mention EABI explicitly.

From what I understand, ELF vs. EABI toolchains would probably affect ARM platforms which has some OS functionality to support shared libraries, etc. Since the NXT Firmware is standalone (and the linker map definition files have to be custom generated), I'm not sure what is the impact?

TC Wan wrote:

TC Wan wrote: From what I understand, ELF vs. EABI toolchains would probably affect ARM platforms which has some OS functionality to support shared libraries, etc. Since the NXT Firmware is standalone (and the linker map definition files have to be custom generated), I'm not sure what is the impact?

Replying to myself....
I tried compiling NxOS (this is the only point of reference I have, I couldn't compile the NXT firmware using arm-eabi-gcc from devKitPro for Mac OS X due to some issues with their pre-built newlib) using arm-eabi-gcc vs. arm-elf-gcc, and it gave the following file sizes:
57675 24 Aug 13:41 tests_rxe.elf_arm-eabi-gcc-4.5.0
59637 24 Aug 13:39 tests_rxe.elf_arm-elf-gcc-4.5.0

The main impact (code size reduction) comes from arm-eabi-gcc generating Thumb instructions, whereas arm-elf-gcc defaults to ARM instructions.

A useful (?) comparison of ARM EABI vs. older (non-EABI) formats is available here

Nicolas Schodet wrote:

John Hansen wrote: Some of my "enhancements" involve refactoring and changes to reduce the code and data size of the resulting build. Are you willing to have those kind of check-ins or would you prefer to limit changes to specific feature additions and bug fixes?
...

This definitely worth the change, I also find that the use of double and float are not consistent.

Moreover, using git will make merging from LEGO easier.

Nicolas Schodet wrote:

TC Wan wrote:
Hmm. I'm still confused regarding the EABI vs. ELF toolchains. Googling doesn't reveal much info.
Prior to this I was building NxOS successfully using the Yagarto ELF toolchain on Mac OSX, and it seems to run fine.
According to ARM Info Center, they've adopted EABI since ~2003 and the BSABI document (ARM IHI 0036B, current through ABI release 2.08 10th October 2008, reissued 28th October 2009) does not mention EABI explicitly.

From what I understand, ELF vs. EABI toolchains would probably affect ARM platforms which has some OS functionality to support shared libraries, etc. Since the NXT Firmware is standalone (and the linker map definition files have to be custom generated), I'm not sure what is the impact?

Quoting http://wiki.debian.org/ArmEabiPort:

With the new ABI, default structure packing changes, as do some default data sizes and alignment (which also have a knock-on effect on structure packing). In particular the minimum size and alignment of a structure was 4 bytes. Under the EABI there is no minimum and the alignment is determined by the types of the components it contains. This will break programs that know too much about the way structures are packed and can break code that writes binary files by dumping and reading structures.

That is the problem with the APCS toolchain (arm-elf-gcc). The NXT firmware wrongly use knowledge about the compiler way of assembling structures. The new EABI match the convention used in the IAR compiler.

Given:

Code: Select all

struct a {
    short int s;
};

IAR and EABI toolchain will make a 2 byte structure, APCS will make a 4 byte structure. The NXT source code wants the IAR convention.

I will add this to the FAQ section to make this clearer.

Nicolas Schodet wrote: Also, arm-elf is being deprecated: http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42100

benco wrote: Hello
Just an idea here, 1/3 of the weight of the firmware source code is about the "try me" programs, in my thought, not needed in "optimized" firmware, even in common use. this is an easy improvement, without touching any compatibility NXT-G/NXC. Am i wrong, or?
Best regards
ben

P.S Nicolas, belle initiative que voilà! j'habite Cachan, tu passes souvent sur la région parisienne?

John Hansen wrote: The programs themselves are not part of the source code - just part of the firmware image as files that you can run. I'm sure neither my IAR-built firmware nor this one includes those files by default. There is a chunk of firmware code that is all about writing on-brick programs. That could be stripped out without breaking the VM. Also the on-board datalogging support could be stripped out as could the manual sensor value inspection features of the user interface. But I don't think that would make a very big difference in code size. Most of the code is in the VM and the various firmware modules that interface with outputs, inputs, buttons, comm of three sorts, i2c support, etc... None of that could be removed without breaking compatibility with NXT-G and NXC.

John Hansen

Spiller wrote: What about the graphics used in those menus? When I skimmed though the source it looked like there was used quite a bit of graphics, but I don't know how much space this actually ends up taking.
The small animation when you turn on the NXT, does it actually do anything in the background? Because I have always found it annoying to wait for it to finish.

Anyway, I would have nothing against removing those from the menu, not because of the file size, but because it is annoying to have random stuff in the menu you will never use anyway.

John Hansen wrote: The graphics would probably only affect the size of the data segment and not the code segment but removing them would reduce the total firmware image size. At least that is how things work in IAR. Since the LEGO MINDSTORMS edition of the IAR compiler is code segment size limited that's the only bit that I have been paying close attention to. It would be a lot of work to strip out all the bits that aren't critical to running code on the VM. But it might be worth creating a version of the firmware that is "streamlined" in that way.

John Hansen

marvin wrote: Hello,

Spiller wrote: What about the graphics used in those menus? When I skimmed though the source it looked like there was used quite a bit of graphics, but I don't know how much space this actually ends up taking.

Looking at the object files there are two big components: The virtual machine and the user interface. So I think by optimizing the user interface the firmware could be shrinked significant. But that depends on the needs of the people.

Perhaps I will try a little bit around but I cannot promise anything...

Bye marvin

Nicolas Schodet wrote: Removing the start-up animation should save 4k just for the bitmaps, so this is a good candidate. I also find the start-up time too long and the brick is not doing anything useful during this interval.

About removing menus, I am not sure if this can easily be done without changing everything. I am not familiar with the menu system yet.

You can evaluate the weight of a feature by looking at the sym file (make sym).

Anyway, I do not want to drop everything, but I could make a configuration system to choose at build time. I could also build binaries with several "flavor" (light, medium, fat...).

Do you feel you need more user space?

Ford Prefect wrote: need more user space?
you can't ever have enough of it!

marvin wrote: Hello,

Nicolas Schodet wrote: Removing the start-up animation should save 4k just for the bitmaps, so this is a good candidate.

I tried yesterday and it saves about 2 k. I think the bitmaps are stored more efficient in the firmware.

Nicolas Schodet wrote: I also find the start-up time too long and the brick is not doing anything useful during this interval.

That can be easily changed in the c_ui.c code. Or maybe in c_ui.h.

Nicolas Schodet wrote: About removing menus, I am not sure if this can easily be done without changing everything. I am not familiar with the menu system yet.

I tried yesterday and I got a slim menu version compiled. But I do not have a simulator and the real brick is at home - not here. So I cannot try the result.

The memory savings is not so much - only about 2 k again. When removing all the related functions and all of the bitmap icons this will increase the memory savings. But I am not sure about the sense of this. It will be a lot of work to get perhaps 10 k of more free memory. This will be bug sensitive - the work has to be done very concentrated.

I think for most of the people the free memory in the brick is enough. For the others 10 k more free memory is even not enough so I suggest to leave this as it is.

Bye marvin

Nicolas Schodet wrote:

marvin wrote: I tried yesterday and it saves about 2 k. I think the bitmaps are stored more efficient in the firmware.

I got my number by looking at the compiler output, I am really surprised you did not save more space. Could you publish your patch?

John Hansen wrote: I think my opinion on this is pretty much in agreement with marvin's. Unless we see a substantial increase in available user flash memory it's probably not worth the time and effort.

John Hansen

afanofosc wrote:I made a change to the enhanced NBC/NXC firmware which enabled a hold at position mode which would (in theory) cause the firmware to hold the motor at the specified tachometer target when it was reached.

afanofosc wrote:Mindsensors has a firmware patch on their website. Here's the contents:

Code: Select all

             case TX_DATA_CLK_HIGH:
             {
               SETClkLow(ChannelNr);
+						  int a; for (a=0; a<60; a++);  // mindsensors:
               if (LowSpeedData[ChannelNr].MaskBit == 0)     				                           //Is Byte Done, then we need a ack from receiver
             {
               SETDataToInput(ChannelNr);                                                             //Set datapin to input

Can anyone comment as to whether and why this would be a good change to make to the firmware? I don't really understand what they are trying to improve.

John Hansen