hw brickbench: Benchmark test for NXT and EV3

[mightor]

Display output tests are impossible right now, since the functionality has not been implemented yet.

I implemented a native shellsort opcode for char arrays and it showed a 1000-fold speed up, when compared to running the same algorithm in ROBOTC. I haven't had time to make it work for other types yet.

= Xander

[HaWe]

Xander,
thank you for this information,
but then please just post your code as-it-is, in the version by which you generated your published results!
The RobotC benchmarks can be updated at any time when your code will be available in an updated version, too.

[mightor]

Helmut,

I've added it to the samples folder that comes with ROBOTC 4.x. I am not able to access that stuff right now, but I will when I get home.

= Xander

Edit: see attached.

[HaWe]

thx for the code, Xander.
at the top it's still announced to be the 3.6 code - is that true?
or is it the 4.0 code?

[mightor]

Oh, I didn't change that part, this is what's shipped with 4.x. I'll fix the header in the benchmark when the next batch of functionality gets added to ROBOTC.

= Xander

[gloomyandy]

Doc the source code for the leJOS version for the EV3 is here:
https://drive.google.com/file/d/0BwTg7x ... sp=sharing

and the NXT here:
https://drive.google.com/file/d/0BwTg7x ... sp=sharing

For some reason you do not seem to be showing the results for test 0-3 for the EV3 version of leJOS.

[HaWe]

thank you very much, Andy!
I'll post it here in pure text for better reading:


hw benchmark for leJOS/Java for NXT:

Code: Select all

import javax.microedition.lcdui.Graphics;

import lejos.nxt.Button;
import lejos.nxt.LCD;


//hw brickbench
//benchmark test for NXT/EV3 and similar Micro Controllers
//PL: leJOS/NXT 0.9.1beta
//Autor: (C) Helmut Wunder 2013
// Ported to leJOS by Andy Shaw.
//freie Verwendung für private Zwecke
//für kommerzielle Zwecke nur nach Genehmigung durch den Autor.
//protected under the friendly Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License
//http://creativecommons.org/licenses/by-nc-sa/3.0/
//version 1.08

// PLEASE NOTE: This is not a good example of a Java program. It is a simple port of a C program. No
// attempt has been made to make use of Java features.

public class MBBenchmark
{
    static long[] runtime = new long[8];
    static int[] a = new int[500], b = new int[500], c = new int[500],
            t = new int[500];

    // --------------------------------------------
    // Mersenne Twister
    // --------------------------------------------

    static int[] y = new int[25];
    static int index = 25 + 1;
    static final int M = 7;
    static final int[] A = { 0, 0x8ebfd028 };

    static int randM()
    {

        if (index >= 25)
        {
            int k;
            if (index > 25)
            {
                int r = 9, s = 3402;
                for (k = 0; k < 25; ++k)
                {
                    r = 509845221 * r + 3;
                    s *= s + 1;
                    y[k] = s + (r >> 10);
                }
            }
            for (k = 0; k < 25 - M; ++k)
                y[k] = y[k + M] ^ (y[k] >> 1) ^ A[y[k] & 1];
            for (; k < 25; ++k)
                y[k] = y[k + (M - 25)] ^ (y[k] >> 1) ^ A[y[k] & 1];
            index = 0;
        }

        int e = y[index++];
        e ^= (e << 7) & 0x2b5b2500;
        e ^= (e << 15) & 0xdb8b0000;
        e ^= (e >> 16);
        return e;
    }

    // --------------------------------------------
    // Matrix Algebra
    // --------------------------------------------

    // matrix * matrix multiplication (matrix product)

    static void MatrixMatrixMult(int N, int M, int K, double A[][],
            double B[][], double C[][])
    {
        int i, j, s; // matrix A: N x M // B: M x K // C: N x K
        for (i = 0; i < N; ++i)
        {
            for (j = 0; j < K; ++j)
            {
                C[i][j] = 0;
                for (s = 0; s < M; ++s)
                {
                    C[i][j] = C[i][j] + A[i][s] * B[s][j];
                }
            }
        }
    }

    // matrix determinant

    static double MatrixDet(int N, double A[][])
    {
        int i, j, i_count, j_count, count = 0;
        double[][] Asub = new double[N - 1][N - 1];
        double det = 0;

        if (N == 1)
            return A[0][0];
        if (N == 2)
            return (A[0][0] * A[1][1] - A[0][1] * A[1][0]);

        for (count = 0; count < N; count++)
        {
            i_count = 0;
            for (i = 1; i < N; i++)
            {
                j_count = 0;
                for (j = 0; j < N; j++)
                {
                    if (j == count)
                        continue;
                    Asub[i_count][j_count] = A[i][j];
                    j_count++;
                }
                i_count++;
            }
            det += Math.pow(-1, count) * A[0][count] * MatrixDet(N - 1, Asub);
        }
        return det;
    }

    // --------------------------------------------
    // shell sort
    // --------------------------------------------

    static void shellsort(int size, int[] A)
    {
        int i, j, increment;
        int temp;
        increment = size / 2;

        while (increment > 0)
        {
            for (i = increment; i < size; i++)
            {
                j = i;
                temp = A[i];
                while ((j >= increment) && (A[j - increment] > temp))
                {
                    A[j] = A[j - increment];
                    j = j - increment;
                }
                A[j] = temp;
            }

            if (increment == 2)
                increment = 1;
            else
                increment = (int) (increment / 2.2);
        }
    }

    // --------------------------------------------
    // benchmark test procedures
    // --------------------------------------------

    static int test_Int_Add()
    {
        int i = 1, j = 11, k = 112, l = 1111, m = 11111, n = -1, o = -11, p = -111, q = -1112, r = -11111;
        int x;
        int s = 0;
        for (x = 0; x < 20000; ++x)
        {
            s += i;
            s += j;
            s += k;
            s += l;
            s += m;
            s += n;
            s += o;
            s += p;
            s += q;
            s += r;
        }
        return s;
    }

    static int test_Int_Mult()
    {
        int x, y;
        int s = 0;

        for (y = 0; y < 2000; ++y)
        {
            s = 1;
            for (x = 1; x <= 13; ++x)
            {
                s *= x;
            }
            for (x = 13; x > 0; --x)
            {
                s /= x;
            }
        }
        return s;
    }

    static float test_float_math()
    {

        float s = (float) Math.PI;
        int y;

        for (y = 0; y < 5000; ++y)
        {
            s *= Math.sqrt(s);
            s = (float) Math.sin(s);
            s *= Math.cos(10.5 * s);
            s = (float) Math.sqrt(s);
            s = (float) Math.exp(s);
        }
        return s;
    }

    static int test_rand_MT()
    {
        int s = 0;
        int y;

        for (y = 0; y < 5000; ++y)
        {
            s = randM() % 10001;
        }
        return s;
    }

    static float test_matrix_math()
    {
        int x;

        double[][] A = new double[2][2], B = new double[2][2], C = new double[2][2];
        double[][] O = new double[3][3], T = new double[3][3];
        int s;

        for (x = 0; x < 250; ++x)
        {

            A[0][0] = 1;
            A[0][1] = 3;
            A[1][0] = 2;
            A[1][1] = 4;

            B[0][0] = 10;
            B[0][1] = 30;
            B[1][0] = 20;
            B[1][1] = 40;

            MatrixMatrixMult(2, 2, 2, A, B, C);

            A[0][0] = 1;
            A[0][1] = 3;
            A[1][0] = 2;
            A[1][1] = 4;

            MatrixDet(2, A);

            O[0][0] = 1;
            O[0][1] = 4;
            O[0][2] = 7;
            O[1][0] = 2;
            O[1][1] = 5;
            O[1][2] = 8;
            O[2][0] = 3;
            O[2][1] = 6;
            O[2][2] = 9;

            MatrixDet(3, O);

        }

        s = (int) (O[0][0] * O[1][1] * O[2][2]);
        return s;
    }

    static int test_Sort()
    {
        int y;
        int[] t = new int[500];

        for (y = 0; y < 50; ++y)
        {
            System.arraycopy(a, 0, t, 0, t.length);
            shellsort(500, t);
            System.arraycopy(b, 0, t, 0, t.length);
            shellsort(500, t);
            System.arraycopy(c, 0, t, 0, t.length);
            shellsort(500, t);
        }

        return y;
    }

    static long test_TextOut()
    {

        int y;

        for (y = 0; y < 20; ++y)
        {
            LCD.clear();

            LCD.drawString("" + 0 + " " + 1000 + "  int_Add", 0, 0);
            LCD.drawString("" + 0 + " " + 1010 + "  int_Mult", 0, 1);
            LCD.drawString("" + 0 + " " + 1020 + "  float_op", 0, 2);
            LCD.drawString("" + 0 + " " + 1030 + "  randomize", 0, 3);
            LCD.drawString("" + 0 + " " + 1040 + "  matrix_algb", 0, 4);
            LCD.drawString("" + 0 + " " + 1050 + "  arr_sort", 0, 5);
            LCD.drawString("" + 0 + " " + 1060 + "  display_text", 0, 6);
            LCD.drawString("" + 0 + " " + 1070 + "  testing...", 0, 7);
        }
        return 99;
    }

    static int test_graphics()
    {
        int x = 1, y;
        Graphics g = new Graphics();;
        for (y = 0; y < 100; ++y)
        {
            g.clear();
            g.drawArc(50, 40, 10, 10, 0, 360);
            g.fillArc(30, 24, 10, 10, 0, 360);
            g.drawLine(10, 10, 60, 60);
            g.drawLine(50, 20, 90, 70);
            g.drawRect(20, 20, 40, 40);
            g.fillRect(65, 25, 20, 30);
            g.drawArc(100, 30, 15, 20, 0, 360);
        }
        return x;
    }


    static void displayValue(int testNo, long testTime, String testName)
    {
        LCD.drawInt(testNo, 2, 0, testNo);
        LCD.drawInt((int) testTime, 6, 3, testNo);
        LCD.drawString(testName, 10, testNo);
    }

    static void displayValues()
    {
        displayValue(0, runtime[0], "int_Add");
        displayValue(1, runtime[1], "int_Mult");
        displayValue(2, runtime[2], "float_op");
        displayValue(3, runtime[3], "randomize");
        displayValue(4, runtime[4], "matrix_algb");
        displayValue(5, runtime[5], "arr_sort");
        displayValue(6, runtime[6], "displ_txt");
        displayValue(7, runtime[7], "graphics");
    }

    static public void main(String[] args)
    {

        long time0;
        int x, y;
        float s;
        int i;

        LCD.clear();
        LCD.drawString("hw brickbench", 0, 1);
        LCD.drawString("(C)H.Wunder 2013", 0, 2);
        LCD.drawString("initializing...", 0, 4);

            for (y = 0; y < 500; ++y)
            {
                a[y] = randM() % 30000;
                b[y] = randM() % 30000;
                c[y] = randM() % 30000;
            }

            LCD.clear();

        time0 = System.currentTimeMillis();
        s = test_Int_Add();
        runtime[0] = System.currentTimeMillis() - time0;
        displayValue(0, runtime[0], "int_Add ");
        System.out.println("int_Add " + runtime[0]);

        time0 = System.currentTimeMillis();
        s = test_Int_Mult();
        runtime[1] = System.currentTimeMillis() - time0;
        displayValue(1, runtime[1], "int_Mult");
        System.out.println("int_Mult " + runtime[1]);

        time0 = System.currentTimeMillis();
        s = test_float_math();
        runtime[2] = System.currentTimeMillis() - time0;
        displayValue(2, runtime[2], "float_op");
        System.out.println("float_op " + runtime[2]);

        time0 = System.currentTimeMillis();
        s = test_rand_MT();
        runtime[3] = System.currentTimeMillis() - time0;
        displayValue(3, runtime[3], "randomize");
        System.out.println("randomize " + runtime[3]);

        time0 = System.currentTimeMillis();
        s = test_matrix_math();
        runtime[4] = System.currentTimeMillis() - time0;
        displayValue(4, runtime[4], "matrix_algb");
        System.out.println("matrx_algb " + runtime[4]);

        time0 = System.currentTimeMillis();
        s = test_Sort();
        runtime[5] = System.currentTimeMillis() - time0;
        displayValue(5, runtime[5], "arr_sort");
        System.out.println("arr_sort " + runtime[5]);

        time0 = System.currentTimeMillis();
        s = test_TextOut();
        runtime[6] = System.currentTimeMillis() - time0;
        System.out.println("text " + runtime[6]);
        LCD.clear();
        displayValues();

        time0 = System.currentTimeMillis();
        s = test_graphics();
        runtime[7] = System.currentTimeMillis() - time0;
        System.out.println("graphics " + runtime[7]);
        LCD.clear();
        displayValues();
        Button.waitForAnyPress();
        LCD.clear();
        y = 0;
        for (x = 0; x < 8; ++x)
        {
            y += runtime[x];
        }
        LCD.drawString("Total ms: " + y, 0, 0);
        LCD.drawString("Benchmark: " + (50000000 / y), 0, 1);
        System.out.println("total ms: " + y);
        System.out.println("benchmark: " + 50000000 / y);
        Button.waitForAnyPress();
        LCD.clear();
    }
}

########################################################################################################

hw benchmark for leJOS/Java for EV3:

Code: Select all

import lejos.hardware.Button;
import lejos.hardware.ev3.LocalEV3;
import lejos.hardware.lcd.GraphicsLCD;
import lejos.hardware.lcd.LCD;

//hw brickbench
//benchmark test for NXT/EV3 and similar Micro Controllers
//PL: leJOS/EV3 0.8.0-alpha
//Autor: (C) Helmut Wunder 2013
// Ported to leJOS by Andy Shaw.
//freie Verwendung für private Zwecke
//für kommerzielle Zwecke nur nach Genehmigung durch den Autor.
//protected under the friendly Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License
//http://creativecommons.org/licenses/by-nc-sa/3.0/
//version 1.08

// PLEASE NOTE: This is not a good example of a Java program. It is a simple port of a C program. No
// attempt has been made to make use of Java features.
public class MBBenchmark
{
    static long[] runtime = new long[8];
    static int[] a = new int[500], b = new int[500], c = new int[500],
            t = new int[500];

    // --------------------------------------------
    // Mersenne Twister
    // --------------------------------------------

    static int[] y = new int[25];
    static int index = 25 + 1;
    static final int M = 7;
    static final int[] A = { 0, 0x8ebfd028 };

    static int randM()
    {

        if (index >= 25)
        {
            int k;
            if (index > 25)
            {
                int r = 9, s = 3402;
                for (k = 0; k < 25; ++k)
                {
                    r = 509845221 * r + 3;
                    s *= s + 1;
                    y[k] = s + (r >> 10);
                }
            }
            for (k = 0; k < 25 - M; ++k)
                y[k] = y[k + M] ^ (y[k] >> 1) ^ A[y[k] & 1];
            for (; k < 25; ++k)
                y[k] = y[k + (M - 25)] ^ (y[k] >> 1) ^ A[y[k] & 1];
            index = 0;
        }

        int e = y[index++];
        e ^= (e << 7) & 0x2b5b2500;
        e ^= (e << 15) & 0xdb8b0000;
        e ^= (e >> 16);
        return e;
    }

    // --------------------------------------------
    // Matrix Algebra
    // --------------------------------------------

    // matrix * matrix multiplication (matrix product)

    static void MatrixMatrixMult(int N, int M, int K, double A[][],
            double B[][], double C[][])
    {
        int i, j, s; // matrix A: N x M // B: M x K // C: N x K
        for (i = 0; i < N; ++i)
        {
            for (j = 0; j < K; ++j)
            {
                C[i][j] = 0;
                for (s = 0; s < M; ++s)
                {
                    C[i][j] = C[i][j] + A[i][s] * B[s][j];
                }
            }
        }
    }

    // matrix determinant

    static double MatrixDet(int N, double A[][])
    {
        int i, j, i_count, j_count, count = 0;
        double[][] Asub = new double[N - 1][N - 1];
        double det = 0;

        if (N == 1)
            return A[0][0];
        if (N == 2)
            return (A[0][0] * A[1][1] - A[0][1] * A[1][0]);

        for (count = 0; count < N; count++)
        {
            i_count = 0;
            for (i = 1; i < N; i++)
            {
                j_count = 0;
                for (j = 0; j < N; j++)
                {
                    if (j == count)
                        continue;
                    Asub[i_count][j_count] = A[i][j];
                    j_count++;
                }
                i_count++;
            }
            det += Math.pow(-1, count) * A[0][count] * MatrixDet(N - 1, Asub);
        }
        return det;
    }

    // --------------------------------------------
    // shell sort
    // --------------------------------------------

    static void shellsort(int size, int[] A)
    {
        int i, j, increment;
        int temp;
        increment = size / 2;

        while (increment > 0)
        {
            for (i = increment; i < size; i++)
            {
                j = i;
                temp = A[i];
                while ((j >= increment) && (A[j - increment] > temp))
                {
                    A[j] = A[j - increment];
                    j = j - increment;
                }
                A[j] = temp;
            }

            if (increment == 2)
                increment = 1;
            else
                increment = (int) (increment / 2.2);
        }
    }

    // --------------------------------------------
    // benchmark test procedures
    // --------------------------------------------

    static int test_Int_Add()
    {
        int i = 1, j = 11, k = 112, l = 1111, m = 11111, n = -1, o = -11, p = -111, q = -1112, r = -11111;
        int x;
        int s = 0;
        for (x = 0; x < 20000; ++x)
        {
            s += i;
            s += j;
            s += k;
            s += l;
            s += m;
            s += n;
            s += o;
            s += p;
            s += q;
            s += r;
        }
        return s;
    }

    static int test_Int_Mult()
    {
        int x, y;
        int s = 0;

        for (y = 0; y < 2000; ++y)
        {
            s = 1;
            for (x = 1; x <= 13; ++x)
            {
                s *= x;
            }
            for (x = 13; x > 0; --x)
            {
                s /= x;
            }
        }
        return s;
    }

    static float test_float_math()
    {

        float s = (float) Math.PI;
        int y;

        for (y = 0; y < 5000; ++y)
        {
            s *= Math.sqrt(s);
            s = (float) Math.sin(s);
            s *= Math.cos(10.5 * s);
            s = (float) Math.sqrt(s);
            s = (float) Math.exp(s);
        }
        return s;
    }

    static int test_rand_MT()
    {
        int s = 0;
        int y;

        for (y = 0; y < 5000; ++y)
        {
            s = randM() % 10001;
        }
        return s;
    }

    static float test_matrix_math()
    {
        int x;

        double[][] A = new double[2][2], B = new double[2][2], C = new double[2][2];
        double[][] O = new double[3][3], T = new double[3][3];
        int s;

        for (x = 0; x < 250; ++x)
        {

            A[0][0] = 1;
            A[0][1] = 3;
            A[1][0] = 2;
            A[1][1] = 4;

            B[0][0] = 10;
            B[0][1] = 30;
            B[1][0] = 20;
            B[1][1] = 40;

            MatrixMatrixMult(2, 2, 2, A, B, C);

            A[0][0] = 1;
            A[0][1] = 3;
            A[1][0] = 2;
            A[1][1] = 4;

            MatrixDet(2, A);

            O[0][0] = 1;
            O[0][1] = 4;
            O[0][2] = 7;
            O[1][0] = 2;
            O[1][1] = 5;
            O[1][2] = 8;
            O[2][0] = 3;
            O[2][1] = 6;
            O[2][2] = 9;

            MatrixDet(3, O);

        }

        s = (int) (O[0][0] * O[1][1] * O[2][2]);
        return s;
    }

    static int test_Sort()
    {
        int y;
        int[] t = new int[500];

        for (y = 0; y < 50; ++y)
        {
            System.arraycopy(a, 0, t, 0, t.length);
            shellsort(500, t);
            System.arraycopy(b, 0, t, 0, t.length);
            shellsort(500, t);
            System.arraycopy(c, 0, t, 0, t.length);
            shellsort(500, t);
        }

        return y;
    }

    static long test_TextOut()
    {

        int y;

        for (y = 0; y < 20; ++y)
        {
            LCD.clear();

            LCD.drawString("" + 0 + " " + 1000 + "  int_Add", 0, 0);
            LCD.drawString("" + 0 + " " + 1010 + "  int_Mult", 0, 1);
            LCD.drawString("" + 0 + " " + 1020 + "  float_op", 0, 2);
            LCD.drawString("" + 0 + " " + 1030 + "  randomize", 0, 3);
            LCD.drawString("" + 0 + " " + 1040 + "  matrix_algb", 0, 4);
            LCD.drawString("" + 0 + " " + 1050 + "  arr_sort", 0, 5);
            LCD.drawString("" + 0 + " " + 1060 + "  display_text", 0, 6);
            LCD.drawString("" + 0 + " " + 1070 + "  testing...", 0, 7);
        }
        return 99;
    }

    static int test_graphics()
    {
        int x = 1, y;
        GraphicsLCD g = LocalEV3.get().getGraphicsLCD();
        for (y = 0; y < 100; ++y)
        {
            g.clear();
            g.drawArc(50, 40, 10, 10, 0, 360);
            g.fillArc(30, 24, 10, 10, 0, 360);
            g.drawLine(10, 10, 60, 60);
            g.drawLine(50, 20, 90, 70);
            g.drawRect(20, 20, 40, 40);
            g.fillRect(65, 25, 20, 30);
            g.drawArc(100, 30, 15, 20, 0, 360);
        }
        return x;
    }


    static void displayValue(int testNo, long testTime, String testName)
    {
        LCD.drawInt(testNo, 2, 0, testNo);
        LCD.drawInt((int) testTime, 6, 3, testNo);
        LCD.drawString(testName, 10, testNo);
    }

    static void displayValues()
    {
        displayValue(0, runtime[0], "int_Add");
        displayValue(1, runtime[1], "int_Mult");
        displayValue(2, runtime[2], "float_op");
        displayValue(3, runtime[3], "randomize");
        displayValue(4, runtime[4], "matrix_algb");
        displayValue(5, runtime[5], "arr_sort");
        displayValue(6, runtime[6], "displ_txt");
        displayValue(7, runtime[7], "graphics");
    }

    static public void main(String[] args)
    {

        long time0;
        int x, y;
        float s;
        int i;

        LCD.clear();
        LCD.drawString("hw brickbench", 0, 1);
        LCD.drawString("(C)H.Wunder 2013", 0, 2);
        LCD.drawString("initializing...", 0, 4);

        // Java on the EV3 uses a JIT compiler, run the test multiple times to
        // allow the results to be stable
        for (int ii = 0; ii < 20; ii++)
        {

            for (y = 0; y < 500; ++y)
            {
                a[y] = randM() % 30000;
                b[y] = randM() % 30000;
                c[y] = randM() % 30000;
            }

            LCD.clear();

            time0 = System.currentTimeMillis();
            s = test_Int_Add();
            runtime[0] = System.currentTimeMillis() - time0;
            displayValue(0, runtime[0], "int_Add ");
            System.out.println("int_Add " + runtime[0]);

            time0 = System.currentTimeMillis();
            s = test_Int_Mult();
            runtime[1] = System.currentTimeMillis() - time0;
            displayValue(1, runtime[1], "int_Mult");
            System.out.println("int_Mult " + runtime[1]);

            time0 = System.currentTimeMillis();
            s = test_float_math();
            runtime[2] = System.currentTimeMillis() - time0;
            displayValue(2, runtime[2], "float_op");
            System.out.println("float_op " + runtime[2]);

            time0 = System.currentTimeMillis();
            s = test_rand_MT();
            runtime[3] = System.currentTimeMillis() - time0;
            displayValue(3, runtime[3], "randomize");
            System.out.println("randomize " + runtime[3]);

            time0 = System.currentTimeMillis();
            s = test_matrix_math();
            runtime[4] = System.currentTimeMillis() - time0;
            displayValue(4, runtime[4], "matrix_algb");
            System.out.println("matrx_algb " + runtime[4]);

            time0 = System.currentTimeMillis();
            s = test_Sort();
            runtime[5] = System.currentTimeMillis() - time0;
            displayValue(5, runtime[5], "arr_sort");
            System.out.println("arr_sort " + runtime[5]);

            time0 = System.currentTimeMillis();
            s = test_TextOut();
            runtime[6] = System.currentTimeMillis() - time0;
            System.out.println("text " + runtime[6]);
            LCD.clear();
            displayValues();

            time0 = System.currentTimeMillis();
            s = test_graphics();
            runtime[7] = System.currentTimeMillis() - time0;
            System.out.println("graphics " + runtime[7]);
            LCD.clear();
            displayValues();
        }
        Button.waitForAnyPress();
        LCD.clear();
        y = 0;
        for (x = 0; x < 8; ++x)
        {
            y += runtime[x];
        }
        LCD.drawString("Total ms: " + y, 0, 0);
        LCD.drawString("Benchmark: " + (50000000 / y), 0, 1);
        System.out.println("total ms: " + y);
        System.out.println("benchmark: " + 50000000 / y);
        Button.waitForAnyPress();
        LCD.clear();
    }

}

########################################################################################################

BTW: Is meanwhile everything finished for the EV3, too, e.g. display benchmarks (or will be still anything changed here), and has the float vs. double thing been fixed intermediately?
What about a java-fw-included sort algorithm?

ps
for the NXT code you wrote " leJOS/EV3 0.8.0-alpha" - but it's probably 0.9.anything, pls CMIIW...?

pps
another question re. the Java JIT compiler:
have the benchs been run once or several times before you took the time?

[gloomyandy]

All of the results are as posted. Things may improve over time, but that is how they are at the moment.

The display results could be improved but it is not a priority for us at the moment. The double/float issue is just a difference between Java and the other systems. Java uses double for most floating point operations (in particular the sin/cos/sqrt/pow etc. functions return doubles), the current benchmark uses floats. This means that the results for the floating point tests are comparing Java using doubles against most of the other systems using floats. On most modern systems this is not a big deal, on the EV3 it may be. On the NXT it certainly is (you can see this in the results). But that is how things are if you use standard Java. I've not made any attempt to use Java features for things like array sorting etc.

Ah yes the version number for the NXT should be 0.9.1beta.

Is there a reason for the question marks against the NXT results?

[HaWe]

version thing fixed,
"?" removed (I simply missed it).

Thanks again!

Ergebnisse / Results (Version 1.17)

Code: Select all

HaWe brickbench © test (time in ms)          NXT         NXT     NXT        NXT           EV3       EV3      EV3      EV3       Arduino     Arduino 
progr.language, API                          NXC       RobotC   leJOS   Toppers C/C++  BCC/gpp C   leJOS   RobotC   C#/Mono     Mega2560      Due   
Firmware / vers.                         JH's EFW       3.6     0.9.1   nxtOSEK BIOS    LegoFW**)  loop(?) 4.0.x  loop#3(-#1)  Sketch 1.5 Sketch 1.5
===================================================================================================================================================
 0 integer add/subtr                        3353       1561      727        29            3      4(-xxx)?    336       2(-15)        129         8
 1 integer multiply/division                6441       1150      463       182           13     11(-xxx)?    589      26(-35)       1219         7
 2 float operations                         1716        413     2543       320           50     45(-xxx)?    226     155(-212)       887        83
 3 Mersenne Tw. PRNG (&,^,>>,<<)            5372       1838       73        55            4      7(-xxx)?   1204      13(-19)        315         8
---------------------------------------------------------------------------------------------------------------------------------------------------
 4 matrix algebra (prod/determ.)            3356        585       68        39           10     60(-xxx)?    325      43(-229)       243        23
 5 int array sort [500]                      332      95879    18756       491          113    139(-xxx)?  20932     172(-299)      1366       171
---------------------------------------------------------------------------------------------------------------------------------------------------
 6 display text     (N/A: penalty=1000) *)   781         32      26        17           28     218(-xxx)?  +1000+?   268(-395)     92016     35347
 7 display graphics (N/A: penalty=1000) *)   384       ? 95?   11480      182           51     371(-xxx)?  +1000+?   185(-475)?   224137    138600
===================================================================================================================================================
execution time                            21,735    101,553   35,117     1,362          272    855(-xxx)? <23600+?      864       316153    174247
===================================================================================================================================================
10 runt. code mem. (score= ln^3)        120k(110)  112k(105)      ?   224k(158)   <64M(1340)       ?          ?     42M(1210)   248k(167) 512k(240)            
11 variable mem.   (score= ln^3)          32k(42)    15k(20)      ?     64k(72)   <64M(1340)       ?     15k(20)    42M(1210)     8k  (8)  96k (95)
12 BT raw master+slave   (score 10x)       4 (40)     2 (20)      ?      2 (20)           0        ?         ?            0            0         0
13   RS485/i2c raw master+slv.(score 10x) 32(320)     2 (20)      ?     32(320)           0        ?         ?            0     110(1100) 220(2200)   
14   BT, +MC I/O rem.protocol (score 50x)      0          0       ?          0            0        ?         ?            0            0         0             
15   RS485, +MC I/O rem.prot. (score 50x)      0          0       ?          0            0        ?         ?            0            0         0
16   USB MC Daisy-Chain       (score 50x)      0          0       ?          0            0        ?         ?            0            0         0
17   WiFi MC I/O rem.protocol (score 50x)      0          0       ?          0            0        ?         ?            0            0         0
18 feat. double=64bit float   (score 100)      0          0      100         0          100       100        0          100            0       100
19 feat.>=4D array/matrix op. (score 100)    100          0       ?          0          100        ?         0          100            0         0
20 Recursions      (score 100)                 0          0       ?        100          100        ?         0          100          100       100
21 Multitthreading (score 100)               100        100      100       100         (100)      100      100          100         (100)      100
22 API complete (sensor+motor+screen)         +          +        +        (+)          ---        +         -          (-)           +         +
23 small IDE made-to-measure  (score 500)    500        500       0          0          500        0       500            0          500       500
===================================================================================================================================================
Brick platform                              NXT         NXT     NXT        NXT         EV3       EV3        EV3         EV3     Ardu.2560   Ardu.Due
progr.language, Firmware                    NXC       RobotC   leJOS     nxtOSEK   BCC/gpp C    leJOS     RobotC     C#/Mono      C/C++       C/C++             
===================================================================================================================================================
speed bench score (50,000,000/exec.time)   2,300      0,485    1,423    36,711      183,823    58,479    >2,100?      57,870       0,158     0,287
environnment bench score                   1,312      0,672   ?0,200     0,790    [---3,080]   ?0,200     0,627?       2,820       2,119     2,487
---------------------------------------------------------------------------------------------------------------------------------------------------
brickbench system benchmark                3,612      1,157?   1,623 ?  37.501  [---186,883]   58,579?   >2,700?      60,690       3,359     2,774
===================================================================================================================================================

link to the most updated charts:
http://www.mindstormsforum.de/viewtopic ... 095#p64772

[HaWe]

still many Lejos and RobotC issues and data missing, but Monobrick has been finished yet by adding display classes (thanks to Vlad Ruzov!).

latest most updated benchmark overview table can be found here:
http://www.mindstormsforum.de/viewtopic ... 095#p64772