Java-Gaming.org Hi !
Featured games (90)
games approved by the League of Dukes
Games in Showcase (727)
Games in Android Showcase (217)
games submitted by our members
Games in WIP (796)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: 1 [2]
  ignore  |  Print  
  Graphics Backend Abstraction  (Read 5456 times)
0 Members and 1 Guest are viewing this topic.
Offline theagentd

« JGO Bitwise Duke »


Medals: 970
Projects: 4
Exp: 8 years



« Reply #30 - Posted 2017-04-12 21:07:49 »

What I'm getting at is, why abstract this, when your "engine" API is already the abstraction? The underlying rendering API - Vulkan or OpenGL (or even DX12!) - is what your engine will be using to turn its own ideas about what it has to render into rendering commands for the specified low-level API. You write a specific plug-in to render what your "engine" contains using the specific API. There's no real point in abstracting the rendering APIs and then using that abstraction to feed to your engine... do it directly. I'm probably not explaining this very well though.

Cas Smiley
I think I understand what you're saying, but I've concluded that it's simply too much work to maintain. IMO, the graphics API is such a vital part of a game engine that the graphics needs to be well-integrated to have good performance. In addition, the graphics APIs actually have very much in common. The optimal strategy is essentially the same on all APIs, but how you accomplish that can differ a bit. The point of that is that regardless of the API, most of your logic will stay the same.

Obviously, the graphics is just a small part of any complete game engine, as there are lots of logic and API-independent functionality that the engine needs. Hence, it makes sense to at least compartmentalize the graphics API as a complete independent module of the game engine. Now, you could just write N different versions of this module, one for each API, but this has a number of big drawbacks. You will end up with a lot of code duplication, as a lot of functionality is almost independent of the API used, to the point where you simply want to call an API function with a different name to accomplish the same thing. Having to rewrite higher level functionality for each API is a lot of wasted time, increase in maintenance and increased risk of bugs. There is also a severe limitation in the extensibility of the engine by the user. There are lots of techniques that require very optimized and tailor-made rendering modules, for example advanced terrain systems, fluid/water rendering, etc, that require a level of control of draw calls that usually cannot be achieved effectively with a general purpose model renderer. By forcing the user to write one version for each graphics API they want to support, you both force them to learn all the APIs they want to support (which is exactly what you want to avoid), which in turn will
encourage users to limit their support to a few specific APIs. This is a major blow to cross platform compatibility, as something as trivial as rewriting a renderer from OpenGL to OpenGL-ES can be prohibitively time consuming. Essentially, such API integration would severely diminish the point of the entire engine.

I have a number of goals with my abstraction:
1. I want to minimize the amount of work I have to put in.
2. I want to minimize the time taken to maintain and add new features to the system.
3. I want to minimize the risk of bugs by avoiding code duplication whenever possible.
4. I want to completely hide the graphics API used to avoid users having to learn all the APIs they want to support and subsequently locking themselves into a single API to save time.
5. I want the user to be able to work very close to the API when writing their own renderers for maximum performance and flexibility, without actually exposing the API being used.

A completely different version of the engine would violate essentially all of these goals. To facilitate point 5, a low-level abstraction of the graphics API is needed, so that the user can take full control of the rendering if needed to accomplish some exotic rendering task. If I'm going to write a low-level abstraction of each API for the user anyway, it makes a lot of sense to base the entire engine on top of that abstraction too. This reduces the amount of code duplication and also forces me to test the abstraction fully, as a bug in the abstractions will show up when I implement the built-in renderers of the game engine.

As an example, consider texture streaming. In OpenGL, this involves creating a separate OpenGL context, mapping buffers, uploading texture data and managing GLsync objects. In Vulkan, it involves a transfer-specialized queue (if available), mapping buffers, uploading texture data and managing fences/semaphores. These concepts are easily abstracted 1-to-1 into a common interface. By abstracting that away at a very low level, I can both write a single texture streaming system based on this common interface and avoid tons of code duplication, and even allow others to write tailor-made texture streaming systems with the same performance as mine for their very own purpose, possibly based on GPU feedback on which texture data is missing, using sparse/virtual textures, etc.


Myomyomyo.
Offline Longor1996
« Reply #31 - Posted 2017-04-13 17:56:15 »

This might be interesting: https://de.slideshare.net/DICEStudio/framegraph-extensible-rendering-architecture-in-frostbite

Offline princec

« JGO Spiffy Duke »


Medals: 904
Projects: 3
Exp: 16 years


Eh? Who? What? ... Me?


« Reply #32 - Posted 2017-04-13 18:20:40 »

I think the concepts in JavaFX have it right.

Cas Smiley

Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline abcdef
« Reply #33 - Posted 2017-04-14 12:11:09 »

In my code i have engine specific enums, so i would build your scene graph (as an example) with no back end specific values. When you then convert this to a data model that your back end renderer will use you then convert the enums to back end specific values. This way you have a distinct separation between rendering and the data model
Offline theagentd

« JGO Bitwise Duke »


Medals: 970
Projects: 4
Exp: 8 years



« Reply #34 - Posted 2017-04-14 12:55:45 »

I think the concepts in JavaFX have it right.

Cas Smiley
Care to elaborate?

In my code i have engine specific enums, so i would build your scene graph (as an example) with no back end specific values. When you then convert this to a data model that your back end renderer will use you then convert the enums to back end specific values. This way you have a distinct separation between rendering and the data model
The entire point of this thread was to figure out how to do that efficiently. =P

Myomyomyo.
Offline abcdef
« Reply #35 - Posted 2017-04-14 14:19:22 »

Apologies theagentd, the OP seemed to be asking about the method rather than asking about the fastest way. Why would speed be an issue for rendering if you do the mapping at the point of initialisation which would mean all the mapping is done prerender?
Offline princec

« JGO Spiffy Duke »


Medals: 904
Projects: 3
Exp: 16 years


Eh? Who? What? ... Me?


« Reply #36 - Posted 2017-04-14 15:24:11 »

Well, JavaFX does not in any way attempt to abstract both DX and GL into a single lookalike API. It resolutely has its own set of constants and commands and a scenegraph type of structure, and it has a completely pluggable back-end on it.

Unity is the same: it presents an API geared towards doing what you want to do, not precisely how to achieve it; it takes care of the stuff at the back end using a pluggable architecture. Even the shaders are in their own special Unity language and compiled to GL, GLES, DX or whatever, depending on the backend.

It seems that pretty much all successful, in-use APIs follow the same paradigm: don't try and find a common ground between the backends you want to support and make the thinnest possible veneer between them; instead, totally hide it all away and present higher-level APIs to the end user.

Cas Smiley

Offline SHC
« Reply #37 - Posted 2017-04-14 20:08:34 »

@princec It certainly is possible to do a low level abstraction, and here is an example of it: https://github.com/livingcreative/xgs

Offline CopyableCougar4
« Reply #38 - Posted 2017-04-15 03:08:45 »

Quote
1. I want to minimize the amount of work I have to put in.
2. I want to minimize the time taken to maintain and add new features to the system.
3. I want to minimize the risk of bugs by avoiding code duplication whenever possible.
4. I want to completely hide the graphics API used to avoid users having to learn all the APIs they want to support and subsequently locking themselves into a single API to save time.
5. I want the user to be able to work very close to the API when writing their own renderers for maximum performance and flexibility, without actually exposing the API being used.

@theagentd It seems like the best solution involves breaking each function/concept (i.e. Textures, Framebuffers, Shaders) into a class, then writing a bunch of static functions. The class would have a static final variable determining which backend, and then the functions would have an if-then-else based on the static final backend variable (to allow for optimizing the code if possible).

This approach:
  • Uses static final variables which probably allows the JVM to optimize each method's if statements
  • Loads the constants statically and only once
  • Allows you to implement features incrementally, keep track of them, and write little duplicate code
  • Is pretty close to the core API
  • Fast initialization of constants, no lookup
  • No objects, low memory overhead

For example,
1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
23  
24  
25  
26  
27  
28  
29  
30  
31  
32  
33  
34  
35  
public class Constants {

   public static final int BACKEND = Integer.parseInt(System.getProperty("backend","0"));

   public static final int POINTS;

   static {
      switch (BACKEND) {
         case 0:
            POINTS = GL11.GL_POINTS;
            break;
      }
   }

   //...

}

public class Texture {

   public static int create() {
      if (Constants.BACKEND == 0)
         return GL11.glGenTextures();
      //...
      return -1;  
   }

   public static void bind(int target, int id) {
      if (Constants.BACKEND == 0)
         GL11.glBindTexture(target, id);
   }

   //...

}

Either wandering the forum or programming. Most likely the latter Smiley

Github: http://github.com/CopyableCougar4
Offline theagentd

« JGO Bitwise Duke »


Medals: 970
Projects: 4
Exp: 8 years



« Reply #39 - Posted 2017-04-16 12:22:07 »

@CopyableCougar4

Nah, that's way too frustrating, error-prone and un-extendable. If you want to add a backend with that system, you need to modify the core classes, meaning that if a user adds their own backend their code will be incompatible with future updates to the main code. It also essentially forces you to have all the backends in the same file, not to mention the huge amounts of switches/if-statements needed in literally every single function.

Using interfaces/abstract classes and having the backends implement them is much more robust, allows for extension without modifying the core classes and is equally fast, since with just one implementation of an interface loaded the function calls can be inlined perfectly.

Myomyomyo.
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline CopyableCougar4
« Reply #40 - Posted 2017-04-17 05:22:41 »

@theagentd Now how would you combine OpenGL (which from Java uses raw values) and Vulkan (which from Java uses structs) together into one API? Would the API manage the memory or is the user responsible for that?

Either wandering the forum or programming. Most likely the latter Smiley

Github: http://github.com/CopyableCougar4
Offline theagentd

« JGO Bitwise Duke »


Medals: 970
Projects: 4
Exp: 8 years



« Reply #41 - Posted 2017-04-17 15:18:14 »

@theagentd Now how would you combine OpenGL (which from Java uses raw values) and Vulkan (which from Java uses structs) together into one API? Would the API manage the memory or is the user responsible for that?
I just hide the structures.

Example:
1. I create a texture, get an abstract texture object back, which either wraps a OGL int target and int textureID, or a Vulkan long pointer.
2. I want to add it to a descriptor set, so I pass it into a descriptor set. The OGL descriptor set casts it into the OGL implementation and gets the target+textureID, while the Vulkan casts to the VK implementation and gets the pointer.
3. OGL backend emulates descriptor sets, adding the texture info to some temp memory which can later be bound. VK sets up a stack allocated structure and updates the descriptor set.

In the case of batchable operations (descriptor set updates, command buffer submissions, swapchain swaps, etc etc etc), the interface also exposes a batch system where you get a batch object, something like this:
1  
2  
3  
4  
5  
CommandSubmissionBatch batch = graphics.getCommandSubmissionBatch();
for(int i = 0; i < 32; i++){
    batch.add(commandBuffers[i]);
}
batch.submit();

In this case, the OGL version will just loop over the command buffers and execute them, while the VK one will actually only submit one draw call for all of them.

Myomyomyo.
Offline cylab

JGO Kernel


Medals: 165



« Reply #42 - Posted 2017-04-17 19:48:46 »

Some questions:
- Who will use the API?
- How likely is it to have anyone add a backend?
- Why would this be desireable or even useful?
- Do you plan to make a living from providing an API?
- Why is Vulkan only not good enough?
- Do you need it for your game?
- Aren't you just procrastinating to hide your creativity block from yourself?

The last one killed all my game dev ambitions years ago by keeping myself occupied with tech details until my money ran out, so don't fall for that. (You might need to work for insurance companies otherwise)

Mathias - I Know What [you] Did Last Summer!
Offline theagentd

« JGO Bitwise Duke »


Medals: 970
Projects: 4
Exp: 8 years



« Reply #43 - Posted 2017-04-17 21:29:32 »

- Who will use the API?
Initially, only me and SkyAphid. We will be basing NNGINE, our 3D engine on it, so WSW and all our future games will use this abstraction. If the engine becomes powerful enough to attract attention, we may attempt to market and license the engine. At them moment, we have no concrete plan of commercializing it. This is mainly for our own purposes.

- How likely is it to have anyone add a backend?
The abstraction is modular. The idea is that you can add multiple backends for different features. The purpose of the abstraction is essentially to make a more powerful and efficient abstraction than LibGDX that we can use to support multiple platforms and APIs, so I do not have any expectations that others will implement their own backends. Regardless, I want it to be possible to support new APIs easily.

- Why would this be desireable or even useful?
We have specific games that are a good fit for Android and HTML5 planned. The ability to switch between APIs is a great feature IMO as it means the games made on top of the abstraction are completely API independent, meaning that porting to a different API is a no-op.

- Do you plan to make a living from providing an API?
We do not plan on it, but keep it open as a potential idea. I do not think it will make me rich.

- Why is Vulkan only not good enough?
Backwards compatibility, especially on Android. Also, seamlessly supporting WebGL is nice (although the user will need to be aware of the limitations of GWT).

- Do you need it for your game?
For our games currently in development, yes-ish. Vulkan support for We Shall Wake would help immensely as the game's CPU scalability is currently only limited by single-threaded API calls. Even just the OpenGL backend I'm developing should give a significant boost to WSW's performance thanks to software command buffers and a dedicated rendering thread, let alone the Vulkan backend. In addition, we have a concrete game with an almost complete design document which would only be viable with Android and GWT support. This abstraction is essentially the convergence of a number of requirements.

- Aren't you just procrastinating to hide your creativity block from yourself?

The last one killed all my game dev ambitions years ago by keeping myself occupied with tech details until my money ran out, so don't fall for that. (You might need to work for insurance companies otherwise)
No, I prioritize work on RF and paid work, developing it in my spare time. I appreciate your concern and I can relate to the money troubles. I am currently living very cheaply and getting by, and I know what awaits me if I end up not being able to support myself as I've worked at a bank writing Java web applications.


I guess it's time for me to speed up my work on this. I've been stuck on some stupid technical details that don't have a clean solution for a little bit too long now. =P

I don't like telling people about WIP in that sense unless it's something concrete that others can appreciate. I don't want praise for plans with no real substance. Hence, I'll be back about this once I have something concrete to show.

Myomyomyo.
Offline Riven
Administrator

« JGO Overlord »


Medals: 1277
Projects: 4
Exp: 16 years


Hand over your head.


« Reply #44 - Posted 2017-04-17 21:59:32 »

Porting being a no-op is, I think, an illusion. Different platforms require different gameplay and different performance trade offs. Whether or not your graphics-engine can seamlessly transition from HTML5/WebGL to multiple dedicated GPUs is a mere side note. The effort that is put into building an VK-like API on top of OpenGL is impressive, but by the time you're ready to publish your game, VK has probably gone mainstream, and WebGL would support it by then.

It all reminds me of some of the efforts Cas and I took upon us, to scale the sprite-engine from the low-budget to high-end GPUs, and seeking fast-paths through outdated drivers and/or hardware. It just got in the way of publishing the games, and that framerate bump wasn't ultimately worth it.

If this technical tinkering takes you 18 months, in that time high-end GPU performance has doubled, but more interestingly: your low-end supported hardware doubled in performance too. To make all that raw power, and super efficient API/engine worth the effort, you need a small army of artists and designers. A one or two man team most likely cannot come close to what the big (and free) engines support, so why not pick an art-style that hides the technical inefficiencies? It's hard enough to turn a tech demo into a polished, playable, purchable game. Very little of game programming is performance sensitive, so why focus all your energy on it?

My $0.02 Kiss

Hi, appreciate more people! Σ ♥ = ¾
Learn how to award medals... and work your way up the social rankings!
Offline ClickerMonkey

JGO Coder


Medals: 26
Exp: 10 years


Game Engineer


« Reply #45 - Posted 2017-04-17 23:29:38 »

I've abstracted the graphics backend in my engine by using a system where entities have a "view" id (generated at runtime) and the graphics backend relates a renderer to that "view".

1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
23  
24  
25  
26  
27  
28  
29  
30  
31  
32  
33  
34  
35  
36  
37  
38  
39  
40  
41  
42  
43  
44  
45  
46  
47  
48  
49  
50  
51  
52  
53  
54  
55  
56  
57  
58  
59  
60  
61  
62  
63  
64  
public interface Renderer<T> {
   public static final int NONE = -1;
   default public Renderer<T> create(T entity) {
      return this;
   }
   public void begin(T entity, GameState state, View view); // <- BEGIN RENDERING
   default public void end(T entity, GameState state, View view) {
      // nothing
   }
   default public void update(T entity) {
      // nothing
   }
   default public void destroy(T entity) {
      // nothing
   }
}


// code found in "entity" type object
   public void setRenderer(int id)
   {
      destroyRenderer();
     
      renderer = Axe.renderers.get( id );
     
      if (renderer != null)
      {
         renderer = renderer.create( this );
      }
   }
   
   protected void destroyRenderer()
   {
      if (renderer != null)
      {
         renderer.destroy( this );
         renderer = null;
      }
   }
   
   public void destroy()
   {
      destroyRenderer();
   }

   public void update(GameState state, View<V> firstView)
   {
      if (renderer != null)
      {
         renderer.update( this );  
      }
   }
   
   public void draw(GameState state, View<V> view)
   {
      if (renderer != null)
      {
         renderer.begin( this, state, view );
         
         // render children
         
         renderer.end( this, state, view );
      }
   }


Where Axe.renderers is an instance of:
1  
2  
3  
4  
5  
6  
7  
8  
9  
10  
11  
12  
13  
14  
15  
16  
17  
18  
19  
20  
21  
22  
23  
24  
25  
26  
27  
28  
29  
30  
31  
public class Registry<T>
{
   private int identifier;
   private T[] items;

   public Registry( T... items )
   {
      this.items = items;
   }

   public int create()
   {
      return identifier++;
   }

   public void register( int identifier, T item )
   {
      if (identifier >= items.length)
      {
         items = Arrays.copyOf( items, identifier + 1 );
      }
     
      items[identifier] = item;
   }

   public <X extends T> X get( int identifier )
   {
      return (identifier >= items.length || identifier < 0 ? null : (X)items[identifier]);
   }

}


When you have a displayable object you define an ID
1  
2  
3  
4  
public class Sprite
{
    public static int VIEW = Axe.renderers.create();
}


And when you load a graphics library they do this:
1  
Axe.renderers.register( Sprite.VIEW, new SpriteRenderer() );


This will vary if you aren't using OOP and are using DOP

The renderer class has a create method which can return a new renderer (per entity instance) or they can all shared a renderer (globally or based on the entity data). You can create different views for each type (ie Sprite) that can render it differently.

The restriction here is that all possible drawable things need to be an object and have a view.

This has been working fine for me!

Offline theagentd

« JGO Bitwise Duke »


Medals: 970
Projects: 4
Exp: 8 years



« Reply #46 - Posted 2017-04-19 20:35:51 »

Porting being a no-op is, I think, an illusion. Different platforms require different gameplay and different performance trade offs. Whether or not your graphics-engine can seamlessly transition from HTML5/WebGL to multiple dedicated GPUs is a mere side note. The effort that is put into building an VK-like API on top of OpenGL is impressive, but by the time you're ready to publish your game, VK has probably gone mainstream, and WebGL would support it by then.

It all reminds me of some of the efforts Cas and I took upon us, to scale the sprite-engine from the low-budget to high-end GPUs, and seeking fast-paths through outdated drivers and/or hardware. It just got in the way of publishing the games, and that framerate bump wasn't ultimately worth it.

If this technical tinkering takes you 18 months, in that time high-end GPU performance has doubled, but more interestingly: your low-end supported hardware doubled in performance too. To make all that raw power, and super efficient API/engine worth the effort, you need a small army of artists and designers. A one or two man team most likely cannot come close to what the big (and free) engines support, so why not pick an art-style that hides the technical inefficiencies? It's hard enough to turn a tech demo into a polished, playable, purchable game. Very little of game programming is performance sensitive, so why focus all your energy on it?

My $0.02 Kiss
Of course, that different platforms require different trade offs is a given. A game designed for a keyboard will not work in app form, and a graphics engine designed for 500GB/sec bandwidth GPUs will not run well on a mobile GPU. However, there most definitely are overlaps in the technical department. For example, allowing the user of an Android tablet to plug in a controller and play the game just like on PC is definitely a plus, and a shitty Intel GPU has more in common with a mobile GPU than a dedicated 400$ GPU from AMD or Nvidia. In other words, alternate ways of controlling the game will be a nice feature to have on mobile/tablets, while a version of the engine tweaked for mobile will work very well as a "toaster mode" setting for weak desktop GPUs too, something we've gotten requests for for WSW.

Note: By "engine", I mean the engine built on top of the abstraction, not the abstraction itself. In the example above, I was specifically referring to using deferred rendering for desktop to take full advantage of the huge amounts of processing power and bandwidth available on desktop to achieve great lighting, while falling back to forward rendering on mobile GPUs where bandwidth is scarce and a 24 byte/pixel G-buffer isn't practical. Again, this is pretty much exactly the same problem that Intel GPUs have as they use system RAM, so using forward rendering on Intel GPUs could easily double or triple performance for us (but only at the absolute lowest settings with minimal lighting of course).

I appreciate your advice regarding technical stuff, but I think that this will be worth it. SkyAphid focuses 100% on developing games using the stuff I write, and I spend most of my time on helping out with that. It's worked out so far. If anything, a solid backend will allow us to work faster when creating the actual game as there's much less to worry about when it comes to performance.

Myomyomyo.
Pages: 1 [2]
  ignore  |  Print  
 
 

 
Archive (299 views)
2017-04-27 17:45:51

buddyBro (486 views)
2017-04-05 03:38:00

CopyableCougar4 (934 views)
2017-03-24 15:39:42

theagentd (955 views)
2017-03-24 15:32:08

Rule (955 views)
2017-03-19 12:43:22

Rule (924 views)
2017-03-19 12:42:17

Rule (925 views)
2017-03-19 12:36:21

theagentd (989 views)
2017-03-16 05:07:07

theagentd (903 views)
2017-03-15 22:37:06

theagentd (698 views)
2017-03-15 22:32:18
List of Learning Resources
by elect
2017-03-13 14:05:44

List of Learning Resources
by elect
2017-03-13 14:04:45

SF/X Libraries
by philfrei
2017-03-02 08:45:19

SF/X Libraries
by philfrei
2017-03-02 08:44:05

SF/X Libraries
by SkyAphid
2017-03-02 06:38:56

SF/X Libraries
by SkyAphid
2017-03-02 06:38:32

SF/X Libraries
by SkyAphid
2017-03-02 06:38:05

SF/X Libraries
by SkyAphid
2017-03-02 06:37:51
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!