Java-Gaming.org    
Featured games (81)
games approved by the League of Dukes
Games in Showcase (487)
Games in Android Showcase (112)
games submitted by our members
Games in WIP (553)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: [1]
  ignore  |  Print  
  Value Types Proposal for Java  (Read 1411 times)
0 Members and 1 Guest are viewing this topic.
Offline actual

JGO Coder


Medals: 23



« Posted 2014-05-02 13:47:47 »

A fairly in depth proposal for value types by John Rose, Brian Goetz, and Guy Steele. Would be nice to see this in Java 9 or even sooner.
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 783
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #1 - Posted 2014-05-02 14:14:18 »

If you can't wait for Java 9 (and Java 10, which will be the performant one Pointing), and want to tinker with a subset of the functionality described in the proposal, you can take a look at my impl:
    http://github.com/riven8192/LibStruct


Value types combined with stack allocation:
1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
// surprise! no object creation
Vec3 struct = new Vec3(x, y, z);
struct.x += 33.77;
boolean a = (vec instanceof Vec3); // false
boolean b = (vec != null); // true
System.out.println(struct); // "<struct@386745454>"

Vec3[] vecs = new Vec3[n+100]; // stack allocated too
// note: no looping to fill the array
vecs[88].x = 56.f;
vecs[3] = vecs[0]; // no copy!
process(vecs, vecs[3], vecs[3].toString());

Vec3[] mapped = StructEnv.map(Vec3.class, byteBuffer);


Stack allocated structs are pointer based (no scalar replacement) on memory regions tied to threads (not ThreadLocals).
Struct arrays use indirection, so you can nullify slots, assign arbitrary structs to slots, pass arrays or array elements as parameters to methods, even if the method parameter is defined to be an Object, like System.out.println(Object), yes, really.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 783
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #2 - Posted 2014-05-02 17:05:12 »

An issue with the Java 9 proposal seems that value types are 'final' (makes sense), so to use them as structs, the compiler (javac) will do boxing/unboxing like with primitives (since java 5). This, to my knowledge, means mass-object-allocation, for which you can only hope Escape Analysis => Scalar Replacement will finally be reliable when Java 9 is released...

I'm not confident it will be usable for games - yet.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline princec

JGO Kernel


Medals: 364
Projects: 3
Exp: 16 years


Eh? Who? What? ... Me?


« Reply #3 - Posted 2014-05-03 08:46:59 »

One of the main uses of value types might be in developing complex memory-efficient structures, which is going to need some way to do "pointers to structs" ... wonder how they might solve that one (haven't read the paper in depth - has anyone else given it a fine examination?)

Cas Smiley

Offline Spasi
« Reply #4 - Posted 2014-05-03 10:49:43 »

The problem with the current proposal seems to be that you can't have pointers/references to value types. So yes, it's a problem for non-linear data structures. More discussion here.
Offline gene9

Senior Member


Medals: 10



« Reply #5 - Posted 2014-05-06 19:35:36 »

Would be nice to see this in Java 9 or even sooner.

There is no way that they will ship this for pre Java 9.

An issue with the Java 9 proposal seems that value types are 'final' (makes sense), so to use them as structs, the compiler (javac) will do boxing/unboxing like with primitives (since java 5).

value types are "persistent" like Strings or Java 8 date/time types. By "use them as structs", you want traditional field modification. Why? Can't you just write your class like any other persistent class? I don't see where you think Java would do boxing on value types.

I'm not confident it will be usable for games - yet.

I guarantee you that these will be usable for games. Point, vector, matrix would all use this. So would general purpose tuple types. An Optional value type would be useful as well.

The problem with the current proposal seems to be that you can't have pointers/references to value types.

One of the main uses of value types might be in developing complex memory-efficient structures, which is going to need some way to do "pointers to structs"

The main purpose of this whole concept is the efficiency you gain by not having a pointer. For the scenarios where you really need pointers, keep using regular class.

The JVM doesn't allow direct pointers/references to primitives like ints and floats without workarounds. That doesn't seem to be a problem.

Data structures like linked lists require recursive references and fundamentally could not use this value type construct.

has anyone else given it a fine examination?

This looks like C# structs with the big difference being the mandatory requirement of (immutable) persistence.

The official Microsoft guidelines for choosing between classes vs structs (http://msdn.microsoft.com/en-us/library/ms229017%28v=vs.110%29.aspx) says only use a struct when:

- It logically represents a single value, similar to primitive types (int, double, etc.).
- It has an instance size under 16 bytes.
- It is immutable.
- It will not have to be boxed frequently.

Microsoft recommends you only use structs with immutable values, but the language doesn't require it. This Java proposal requires persistent immutability. The downside of having large structs is that pass-by-value copy operations become expensive. When the runtime has a guarantee of immutability, it can internally pass by reference instead.

Persistent data structures are inherently thread safe and logically easy to work with, but I don't see what other advantage that requirement provides. There must be some advantage or the Java proposal wouldn't require it, but I don't see what that is.

Even the relatively new tuple classes introduced in C# 4.5 uses regular classes, not structs, just like you would do in current versions of Java. I wonder why?

Also, even third party "Option" types in C# all seem to use classes, not structs. And Scala with its limited value type support, doesn't use value types for Option and implements Option as a regular class.




Online Roquen
« Reply #6 - Posted 2014-05-06 19:46:25 »

The main purpose of this whole concept is the efficiency you gain by not having a pointer. For the scenarios where you really need pointers, keep using regular class.

The JVM doesn't allow direct pointers/references to primitives like ints and floats without workarounds. That doesn't seem to be a problem.
The lack of ability to perform limited pointer arithmetic is one of the biggest problems for the JVM.
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 783
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #7 - Posted 2014-05-06 19:47:18 »

Stack allocated and/or mapped mutable datastructures are invaluable for high performance games, where copying all your vertex attributes into a VBO is a waste of CPU cycles. If you map structs into the same memory region the 'copy' is free. I also can't see myself doing vector math with value types... for every operation you have to figure out how to get the value(s) back to the callsite(s). Without mutable state you'll feel severely restricted, and the game related use cases disappear quickly.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline gene9

Senior Member


Medals: 10



« Reply #8 - Posted 2014-05-06 19:56:17 »

Stack allocated and/or mapped mutable datastructures are invaluable for high performance games

Can you point to an example of such a data structure?

I also can't see myself doing vector math with value types... for every operation you have to figure out how to get the value(s) back to the callsite(s). Without mutable state you'll feel severely restricted, and the game related use cases disappear quickly.

Working with persistent vectors shouldn't be any more difficult than working with immutable ints/floats/doubles. Can you show me a simple code example that illustrates the complexity with persistent types?
Online Roquen
« Reply #9 - Posted 2014-05-06 20:00:42 »

Randomly choose a paper on cache-oblivous data structures.
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline junkdog
« Reply #10 - Posted 2014-05-06 21:28:43 »

Derailing back to Riven's original reply.

I only very briefly skimmed the outer layers, but LibStruct looks really interesting!

When you say stack allocation, does that mean that you do the equivalent of scalar replacement? If so, are there any clever tricks to ensure that the intermediate representation doesn't grow too big for the JIT to optimize - apart from fiddlling with XX:MaxInlineSize etc?

I'm assuming the Vec3 class in the code snippet is one's own and not something out of an external library - or is it possible to allocate "normal" classes on the stack too (via external configuration or similar)? Would be super-useful, though maybe not plausible.

artemis-odb: bugfixing and performance optimized fork of artemis ES
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 783
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #11 - Posted 2014-05-06 21:30:26 »

No, there is no scalar replacement. That's exceptionally hard to accomplish. Everything is based on pointers, just like with 'normal' stack allocation in C, where the stack is just another region of main memory.

And yes, you can define your own structs, just like the readme.md shows on GitHub.
So far it doesn't support nested structs yet (shouldn't be hard) but it will probably never support flattened nested structs.

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline junkdog
« Reply #12 - Posted 2014-05-06 21:40:10 »

Sorry, by "normal" classes I meant classes that came from an external library; would be neat to if it were possible to retrofit those too but that might be much harder.


artemis-odb: bugfixing and performance optimized fork of artemis ES
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 783
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #13 - Posted 2014-05-06 21:42:28 »

There is actually support for that, eventhough annotations are out of the question with external code.

Instead of structdef.txt looking like this:
1  
my.clazz.MyStruct


feed it this:
1  
2  
3  
4  
their.clazz.TheirStruct SIZEOF 20
their.clazz.TheirStruct FIELD x F 0
their.clazz.TheirStruct FIELD y F 4
their.clazz.TheirStruct FIELD z F 8
(F = float)

It's very unlikely that a 3rd party class would be suitable to be used as a struct. There is just theoretical support ^.^

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline junkdog
« Reply #14 - Posted 2014-05-06 21:54:43 »

Heh, must experiment with this during the weekend!

It's very unlikely that a 3rd party class would be suitable to be used as a struct. There is just theoretical support ^.^

I was thinking along the lines of libgdx's vector classes. Right now I'm using my own, more constrained struct-like approach, but it results in having to re-implement a lot of the math utility classes.

artemis-odb: bugfixing and performance optimized fork of artemis ES
Offline princec

JGO Kernel


Medals: 364
Projects: 3
Exp: 16 years


Eh? Who? What? ... Me?


« Reply #15 - Posted 2014-05-07 08:23:03 »

The great is the enemy of the good and all that... I can't help but thinking that mapped objects in bytebuffers would have given us everything we needed, and escape analysis and scalar replacement would have given everybody else what they really needed, and it would have been much more easy for them to implement. libstruct looks like a more useful solution to more people than the value types proposal in many ways.

Cas Smiley

Online Roquen
« Reply #16 - Posted 2014-05-07 08:27:07 »

I've only very quickly skimmed it.  It looks like the motivation here is when we hit many-cores (hundreds, thousands) and is targeting functional style multi-threaded as a scalability solution...but maybe I'm misreading it.
Offline pjt33
« Reply #17 - Posted 2014-05-07 13:08:07 »

Microsoft recommends you only use structs with immutable values, but the language doesn't require it.

That's an understatement. C# makes it impossible, because structs always have an implicit no-args constructor which doesn't initialise any of the fields (even if you define a no-args constructor which does initialise them). I've was bitten by that at least once.
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 783
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #18 - Posted 2014-05-07 13:20:14 »

<offtopic>

In my LibStruct I run the actual constructor (not just a default constructor), but disabled running the (default!) struct-constructors when allocating arrays and mapping structs to buffers.

In the case of mapping structs to buffers, you really don't want the constructors to run (as they'd overwrite the mapped memory) but I still have to decide whether running default constructors when allocating arrays is the right thing to do.

1  
2  
3  
4  
new Vec3();            // stack-alloc, run static method: <_init_>(I)V
new Vec3(3, 5.6f, 7);  // stack-alloc, run static method: <_init_>(IFFF)V
new Vec3[n]            // stack-alloc, no constructors are run (yet...)
StructEnv.map(Vec3.class, bb) // no constructors are run


</offtopic>

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Offline delt0r

JGO Knight


Medals: 27
Exp: 18 years


Computers can do that?


« Reply #19 - Posted 2014-05-07 13:43:43 »

I must admit that i don't really see me using anything outside mapped objects in bytebuffers. I also would expect more C like behaviour with something called a struct. ie no Constructors ever. But then if there are references in the struct?

I have no special talents. I am only passionately curious.--Albert Einstein
Offline junkdog
« Reply #20 - Posted 2014-05-07 13:50:58 »

In C++ (and C#, I believe) a struct is only superficially/syntactically different from a class; so constructors and initializer lists work.

artemis-odb: bugfixing and performance optimized fork of artemis ES
Online Roquen
« Reply #21 - Posted 2014-05-08 08:01:09 »

A recent comment by John Rose on the dev-list makes it look like the "plan" is for multiple solutions...specifically that "Arrays 2.0" and "Value Types" are both being worked on.

http://openjdk.5641.n7.nabble.com/Unsafe-for-array-access-td187718.html
Offline Riven
« League of Dukes »

JGO Overlord


Medals: 783
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #22 - Posted 2014-05-08 11:13:42 »

Eh, I subscribed to that mailinglist and I read that post yesterday, but my interpretation is that the subject it discusses is not related to array handling. It discusses optimisations to make volatile array access through Unsafe as fast as regular array access, and thereby providing fast volatile I/O on arrays.

Can you elaborate on how this subject overlaps with value-types / structs?

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings
Online Roquen
« Reply #23 - Posted 2014-05-08 11:53:37 »

Both from John Rose:
Quote
We need an intrinsic like this for Arrays 2.0. Might as well do it now, since we have an immediate application.
Quote
It's not a class.  (Some day a value type!)
Value types is his baby, but seems to be approving of the Arrays 2.0 proposal at the same time.
Pages: [1]
  ignore  |  Print  
 
 

 

Add your game by posting it in the WIP section,
or publish it in Showcase.

The first screenshot will be displayed as a thumbnail.

TehJavaDev (12 views)
2014-08-28 18:26:30

CopyableCougar4 (24 views)
2014-08-22 19:31:30

atombrot (37 views)
2014-08-19 09:29:53

Tekkerue (30 views)
2014-08-16 06:45:27

Tekkerue (29 views)
2014-08-16 06:22:17

Tekkerue (18 views)
2014-08-16 06:20:21

Tekkerue (28 views)
2014-08-16 06:12:11

Rayexar (65 views)
2014-08-11 02:49:23

BurntPizza (41 views)
2014-08-09 21:09:32

BurntPizza (33 views)
2014-08-08 02:01:56
List of Learning Resources
by Longor1996
2014-08-16 10:40:00

List of Learning Resources
by SilverTiger
2014-08-05 19:33:27

Resources for WIP games
by CogWheelz
2014-08-01 16:20:17

Resources for WIP games
by CogWheelz
2014-08-01 16:19:50

List of Learning Resources
by SilverTiger
2014-07-31 16:29:50

List of Learning Resources
by SilverTiger
2014-07-31 16:26:06

List of Learning Resources
by SilverTiger
2014-07-31 11:54:12

HotSpot Options
by dleskov
2014-07-08 01:59:08
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!