[DRAFT] GDExtension: Undeprecate array_ref and add dictionary_ref

[dsnopek]

What problem(s) does this PR solve?

• Fixes call function creating an empty array before calling GDE method pointer #119440

Additional information

Marked as DRAFT for now, because I haven't had a chance to test this with godot-cpp yet, and I'm not 100% sure it's the solution we should persue

[Ferinzz]

Performance test

usage

As a means to replace the Array referenced in the return value of a ClassMethodPtrCall. ie an exposed getter method.

methodology

run for 1000 frames. In each frame call a series of GDE methods (same method(s) repeated 25 times) in order to dereference and assign an array to an initialized TypePtr. Begin a stopwatch before the GDE methods were called, stop it after they are completely called. Assign the time in nanoseconds to an array.

Two methods were used

1. call array_ref
2. call Destroy followed by the Create1 copy constructor

Stopwatch has100 nanosecond precision. sub 100 it would report 0
time is reported in nanoseconds

using Godot release version 4.6
OS Windows 11
language Odin
compiled with -o:speed
launch via cli C:\Godot\Godot_v4.6-release.exe --path ./TopDown

code

//Called each frame through a getter from GDScript
get_tPtr_Array_passthrough :: proc "c" (method_userdata: rawptr, p_instance: GDE.ClassInstancePtr, args: [^]rawptr, r_return: ^Array) {
    ret: Array
    get_passthrough(method_userdata, p_instance, args, &ret)
    watch: time.Stopwatch
    if counter < 1000 {
        time.stopwatch_start(&watch)
        //One of the two below repeated 25 times
        gdAPI.Packed_Array_Utils.ArrayRef(r_return, &ret)
        //Destroy(r_return)
        //GDW.Array_M_List.Create1(r_return, {&ret})
        time.stopwatch_stop(&watch)
        _, _, s, nano:= time.precise_clock(watch)
        times_baseline[counter]=nano
        if nano > slowest do slowest = nano
        if nano < fastest do fastest = nano
        counter+=1
    } else {
        if timer_once == false {
            context = runtime.default_context()
            fmt.println(times_baseline)
            k:int
            for i in times_baseline {
                k+=i
            }
            fmt.println("average", k/1000)
            fmt.println("slowest", slowest)
            fmt.println("fastest", fastest)
            timer_once = true
        }
    }

timer baseline

average 17
slowest 100
fastest 0

average 18
slowest 100
fastest 0

array_ref

gdAPI.Packed_Array_Utils.ArrayRef(r_return, &ret)
ArrayRef is InterfaceArrayRef

average 452
slowest 1300
fastest 200

average 429
slowest 1400
fastest 300

average 500
slowest 19800
fastest 200

average 464
slowest 2800
fastest 200

average 578
slowest 15200
fastest 300

average 589
slowest 140700
fastest 300

average 484
slowest 26800
fastest 300

destroy create

        Destroy(r_return)
        GDW.Array_M_List.Create1(r_return, {&ret})

Create1 is the builtin constructor from GetPtrConstructor(.ARRAY, 1)

average 1348
slowest 27500
fastest 1200

average 1360
slowest 28100
fastest 1200

average 1300
slowest 5900
fastest 1200

average 1342
slowest 7500
fastest 1200

average 1348
slowest 7000
fastest 1200

average 1434
slowest 63000
fastest 1200

average 1410
slowest 21300
fastest 1200

short conclusion

This is a very rudimentary test which only looks at one use case. This does not look at a case where the source or destination is a null value. There are several checks to exit earlier from the = operator overload for those.

On average array_ref is more than 2x faster. Taking the fastest times the difference is 4-6x.
I'm not sure where the outliers are coming from, could be due to the ref counting itself?

If there is no other direct equivalent via GDE interface I will be using array_ref for at least this use case.

[Bromeon]

There are several problems with your measurement:

    time.stopwatch_start(&watch)
    //One of the two below repeated 25 times
    gdAPI.Packed_Array_Utils.ArrayRef(r_return, &ret)
    //Destroy(r_return)
    //GDW.Array_M_List.Create1(r_return, {&ret})
    time.stopwatch_stop(&watch)

1. You reuse the same array. After the first run, the reference will already be assigned and no work needs to be done anymore. This doesn't reflect a real use case.
2. You compare only array_ref vs. a full destroy + create cycle. Of course that's much faster, but it's not expressive because you're not doing isolated array_ref calls in reality, you combine them with another FFI call.

TLDR: you'd need to measure two end-to-end scenarios that do the same thing (e.g. fetch and assign an array).
It's also important because this will tell us how many % the array_ref approach gains over an alternative.

Btw, you can use ```odin tags for syntax highlighting 🙂

[Ferinzz]

Thanks for the feedback! I even mentioned the early return in the conclusion 🤦
I did the 25x repetition because the array_ref was too fast to capture on the stopwatch some frames. I could prep 25 different arrays and pass those through the two different methods so that it's different arrays each time.

You compare only array_ref vs. a full destroy + create cycle. Of course that's much faster, but it's not expressive because you're not doing isolated array_ref calls in reality, you combine them with another FFI call.

Only using Create1 causes a mem leak when the destination array is already initialized. What does godot-cpp and rust do differently? I do want to get this comparison right.