Java-Gaming.org    
Featured games (91)
games approved by the League of Dukes
Games in Showcase (576)
games submitted by our members
Games in WIP (498)
games currently in development
News: Read the Java Gaming Resources, or peek at the official Java tutorials
 
    Home     Help   Search   Login   Register   
Pages: [1]
  ignore  |  Print  
  2D normal maps  (Read 3596 times)
0 Members and 1 Guest are viewing this topic.
Offline Nate

JGO Kernel


Medals: 128
Projects: 3
Exp: 14 years


Esoteric Software


« Posted 2013-10-31 13:32:25 »

I would have necro'ed davedes' topic here:
http://www.java-gaming.org/topics/glsl-using-normal-maps-to-illuminate-a-2d-texture-libgdx/27516/view.html
Too bad I can't post to it, at least to link to this topic since it is related.

Anyway, neat stuff in that thread! I played around with his shader and a rough normal map to apply lighting to a Spine skeleton:



Code (with davedes' permission) is here:
https://github.com/EsotericSoftware/spine-runtimes/blob/master/spine-libgdx/test/com/esotericsoftware/spine/NormalMapTest.java

Runnable JAR is here:
http://esotericsoftware.com/spine/files/demos/spine-libgdx-normals.jar

Sort of related, this is neat:
http://www.spritelamp.com/

Online matheus23

JGO Wizard


Medals: 97
Projects: 3


You think about my Avatar right now!


« Reply #1 - Posted 2013-10-31 13:39:50 »

It looks cool but somewhat strange around the ears and the chin.

You said this is a rough normal map, how rough is it? And how would it look with a better normal map? Smiley

See my:
    My development Blog:     | Or look at my RPG | Or simply my coding
http://matheusdev.tumblr.comRuins of Revenge  |      On Github
Offline Shiu

Junior Member


Medals: 2



« Reply #2 - Posted 2013-10-31 13:41:53 »

The normal map is generated directly from the diffuse texture. Since the diffuse texture already has lighting painted on to it, it doesn't work very well. If the colors were flat the lighting would look far better and the "strangeness" would disappear around areas like the ears and chin.
Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline Nate

JGO Kernel


Medals: 128
Projects: 3
Exp: 14 years


Esoteric Software


« Reply #3 - Posted 2013-10-31 13:45:09 »

Actually this particular normal map wasn't generated AFAIK, it was drawn by the girlfriend of the guy doing the SpriteLamp stuff, and presumably processed with SpriteLamp. She did it really quick and rough and insisted that I mention she makes no guarantees to the quality. Smiley Maybe she'll do a better one if she finds out I'm showing this around! Wink

It should look really good with a better one. See the SpriteLamp link above for some nice examples. Here it is, packed:




Offline Dinaroozie

Junior Newbie




Sprite Lamp


« Reply #4 - Posted 2013-10-31 14:27:24 »

Hey all, Sprite Lamp guy here. Damn cool seeing some normal maps from my software in conjunction with skelmesh animation!

Nate: You are correct, this was processed by Sprite Lamp. I actually think the normals mostly came out okay, except for the head (as matheus23 mentioned), because it has a big old hard edge running right down the side. That should be a pretty easy fix, except that it's the middle of the night here (east coast of Australia) and the artist in question is asleep so the fix will have to wait. Smiley

Also - and I'm coming from a position of some ignorance here, and squinting through tired eyes at foreign code, so I could be wrong - but I suspect there might be a bug in that normal map demo. You can see there's a discontinuity in the lighting at some of the joints, particularly the character's right (camera-facing) elbow. I suspect what's going on is that the normal value is being read directly from the map, but not correctly rotated in the pixel shader. This will result in roughly correct lighting on body parts that are rotated roughly in the same way as they appear in the texture atlas (like the head), but the further from that orientation they deviate, the worse it will get. Normally this is fixed by adding full normal/tangent information to the vertices of a mesh and multiplying by the TNB matrix (I think that's what they call it?). Because in this case we're working in 2D, there might be some clever way to avoid that. Again, this is just my suspicion having looked at the code and the screenshot, but I could be way off base - apologies to the person who wrote the code if I'm full of crap!

Okay, I'm new here so I don't know how fast this forum moves, but in case anyone responds to me - I apologise for not getting back to you in a timely fashion, because for the moment, I have to go and sleep. Smiley

 ~ Finn
Offline Nate

JGO Kernel


Medals: 128
Projects: 3
Exp: 14 years


Esoteric Software


« Reply #5 - Posted 2013-10-31 14:47:54 »

Ah, right. I disabled rotation when packing the images and it is looking better. Runnable JAR updated!


Offline davedes
« Reply #6 - Posted 2013-10-31 15:21:26 »

Sweet.

I haven't looked at SpriteLamp. But I think a tool that allows artists to paint normal maps for 2D sprites (kind of like ZBrush painting) would be awesome.

Or how bout this: a tool/plugin for 3D software that will export your untextured 3D character from a certain camera angle. The plugin exports two sprite sheets: your character's normals (no need to paint them -- and they are accurate and smooth) and your character's wire/clay blueprint that you can then texture (for diffuse/spec/etc). It exports each 3D bone as a 2D sprite, suitable for software like Spine to make use of.

Offline Nate

JGO Kernel


Medals: 128
Projects: 3
Exp: 14 years


Esoteric Software


« Reply #7 - Posted 2013-11-01 18:27:02 »

I imagine it wouldn't be too hard to use a 3D tool like that.

New diffuse and normal maps from Finn:

Runnable JAR is updated too.

Offline Dinaroozie

Junior Newbie




Sprite Lamp


« Reply #8 - Posted 2013-11-02 01:59:35 »

So, turns out it's difficult to do artwork for a tool without being familiar with the tool. Smiley We're gradually improving the normal maps at our end and sending them through to Nate but there are still some issues - for instance, we didn't realise that the 'eyes open' and 'eyes closed' frames needed to be done separately, so in the runnable JAR that's up there now, the area around Spineboy's eyes are weird and flat.

matheus23: Since you asked what it would look like with a less rough normal map, I've taken the liberty of assembling the normal/diffuse maps we do have into a neutral pose and exporting a lighting preview via Sprite Lamp. This includes a few fancy shader effects, but nothing that would make it incompatible with Spine animation. Smiley Unfortunately, right at the moment I don't have the time to fully implement these effects with Spine's runtimes, but I'm starting to think it would make a cool stretch goal for my kickstarter.



davedes: Sprite Lamp is definitely more like the first thing you mentioned - the whole reason for its existence is to make dynamic lighting possible while preserving styles unique to 2D art, such as visible brushstrokes or pixel art. In other words, it's a way of creating normal maps in specific styles that are hard to do with 3D modelling. Of course, a rendering implementation involving normal maps doesn't care about such things - it doesn't matter whether the normal maps come from Sprite Lamp, a depth map, a mesh, or anything else you can think of. One of the key design decisions with Sprite Lamp was that it was for processing images, and how you create those images is up to you - I figured artists are so used to using Photoshop or ArtRage or whatever that trying to make an interface for them to paint in that won't be frustratingly limited would be out of my scope. Smiley

Also, am I right in thinking you're the one I should thank for implementing normal mapping in the Spine runtime? If so, thanks! I have a feeling at some point I'll be working with that code. I'm wondering, do you have any thoughts on what I mentioned up thread, regarding correct rotation of normal maps? Having looked at the JAR posted here, I strongly suspect that that problem is present (that is, the normals aren't being correctly rotated with the underlying geometry). The best way to see the issue is to position the light about here:

See how given the position of the light source (that white dot I drew in), Spineboy's right forearm ought to be underlit, but instead the top is lit?

Anyway, I'm just wondering if you think I might be right about that - not asking you to rework the code or anything! If the problem is what I think it is, the solution is likely to be nontrivial anyway - especially once Spine gets soft skinning and freeform deformation added, which will serve to further complicate things. I'm just trying get a feel for what I'm up against if the time comes. Smiley

 ~ Finn
Offline Nate

JGO Kernel


Medals: 128
Projects: 3
Exp: 14 years


Esoteric Software


« Reply #9 - Posted 2013-11-02 09:37:45 »

You're right, the normals aren't rotated as the skeleton animates. That would be quite cool!

The new diffuse and normal images can be seen here:




The parts that don't have proper normals aren't very important for the skeleton, so probably not worth doing for this example.

Games published by our own members! Check 'em out!
Legends of Yore - The Casual Retro Roguelike
Offline davedes
« Reply #10 - Posted 2013-11-02 17:59:02 »

Hmm.. Rotating the normals. You could rotate the texture coords before sampling from the normal map, but its best to stay away from dependent texture reads and it might lead to some weird stuff with wrapping/filtering. It should be better to rotate the vec3 normal after sampling. Either way you would need to send a rotation for each bone -- as a uniform or vertex attribute. And presumably an offset, for rotation anchor point.

I wonder whether the effect would be noticeably improved, and worth the performance cost.

That sprite lamp preview looks pretty darn amazing. And you can really easily colorize the diffuse map since its so flat.

Online Danny02
« Reply #11 - Posted 2013-11-02 19:24:46 »

@davedes you have to do the rotating or the normals are completely wrong. You might not notice it in this example, but just flip the character on the y-axis and the light is coming from the wrong direction. I mean it is quite common for a ingame char to be able to move from left to right and the other way around.
Offline Dinaroozie

Junior Newbie




Sprite Lamp


« Reply #12 - Posted 2013-11-03 07:40:51 »

Danny02 speaks the truth, unfortunately. If Spineboy was rotated 180 degrees, the unrotated normals would make his head look concave, for instance. It'd get weird.

Fortunately, the maths for this is pretty well-trodden ground, because it's a requirement of doing normal mapping in 3D applications (or even just vertex lighting, from back in the day). The long and the short of it is, you have to multiply the normals by the inverse of the transpose of the model view matrix, and then everything works out okay. Smiley You do this after extracting the normal value from the texture, so you can avoid any texture-dependent lookup shenanigans - the only price to pay is a vertex-matrix multiplication in the pixel shader. I think this is possible even on phone GPUs these days, although perhaps just barely, especially with pixel density being what it is these days.

Things do get a bit hairier if you want to deal with stuff like soft skinning or other per-vertex deformation - basically, all the stuff they're going to add to Spine after this next Kickstarter. Smiley

Also, thanks! I'm glad you like the preview. I totally agree about the diffuse map - having it be separate from the normal map allows really easy recolouring of characters and the like, like a super powerful palette swap.

 ~ Finn
Online Danny02
« Reply #13 - Posted 2013-11-03 10:13:57 »

you can get around the need of a matrix multiplication in the fragment shader if you define a tangent and bi-tangent vector in the vertex shader.

you would have to choose these two vectors in the vertex shader in a way so that they point in the uv directions (tangent in u, and bi-tangent in v).
I did provide in my 3D application only the tangent as a vertex attribute and calculated the bi-tangent from the cross product.

The normal Matrix which is described by Dinaroozie have to be applied on the normal, tangent and bi-tangent vector before they are given over to the fragment shader. If you say that your base normal is (0,0,1) you can spare out the calculations for the normal(as in my code). Also, the normal matrix must only be calculated from the transpose inverse if the model matrix wasn't homogeneous (no shearing or non uniform scaling). If it is, you can just take the rotation part(top left 3x3).

1  
2  
3  
4  
5  
6  
7  
/fragment 
vec3 T = normalize(tangent);
vec3 B = normalize(bitangent);
vec3 tn = normalize(texture(normal_sampler, texcoord).rgb*0.5-0.5);
vec3 N = T * tn.x  + B * tn.y;//normaly also  "normal * tn.z", but I guess that the normal is (0,0,1) in your case
N.z += tn.z;
N = normalize(N);
Offline cylab

JGO Knight


Medals: 34



« Reply #14 - Posted 2013-11-03 10:28:23 »

cant you just rotate the lightsource per bone instead to "simulate" the effect you'll get from rotating the normals?  

Mathias - I Know What [you] Did Last Summer!
Offline Nate

JGO Kernel


Medals: 128
Projects: 3
Exp: 14 years


Esoteric Software


« Reply #15 - Posted 2013-11-03 10:32:03 »

That would break batching the bone images.

Offline cylab

JGO Knight


Medals: 34



« Reply #16 - Posted 2013-11-03 12:01:35 »

hmm, would it? I thought of passing the rotated lightsource as vertex attribute. compute it per bone and pass the same for every attached vertex, but you may also rotate it in the vertex shader when you pass the bone matrix. of course this requires that you can pass additional vertex attributes per batched image...

Mathias - I Know What [you] Did Last Summer!
Offline Dinaroozie

Junior Newbie




Sprite Lamp


« Reply #17 - Posted 2013-11-03 12:24:55 »

Danny02: Wouldn't four normalize calls do more performance damage than a matrix multiply though? Admittedly, the solution I proposed isn't as versatile when it comes to deformation and the like. Also admittedly, I don't really know what I'm talking about when it comes to shader optimisation so maybe I should just be quiet. Smiley

Nate: I think cylab is describing just taking the light's position and getting it in tangent space, rather than getting the normals and putting them in world space. I think that solution should work, and it might maybe even play nice with soft skinning...? But yeah I don't think the images would need to be changed for that, so it shouldn't mess up batching.

This conversation has also made me realise that an atlas tool that handles normal maps without knowing that that's what it's doing might screw things up (by rotating them). I have to give this some thought, because it sounds like something Sprite Lamp ought to deal with intelligently.
Online Danny02
« Reply #18 - Posted 2013-11-03 13:56:53 »

@Dinaroozie, yes that is right. I compared the two shaders and the one with normalization takes 2 more cycles on AMD cards then the matrix multiplication. On the other hand did I take that code directly from my 3D stuff. With some thinking I came up with the following code which doesn't need any normalization and is faster than the matrix calculation based on AMD ShaderAnalyzer.

1  
2  
3  
4  
5  
vec2 tangent = ...;//normalized vertex input
vec3 normal = ...;//normalized texture read

vec2 bi_tangent = vec2(-tangent.y, tangent.x);//have to see if the texture orientation is correct
vec3 N = vec3(tangent * normal.x + bi_tangent * normal.y, normal.z);//no normalization needed if both normal and tangent are normalized


The tangent should be provided as a normalized vector in the vertex data, it might be needed to normalize it in the fragment shader for skinning. In the case of skinning one would have even more problems with the matrix approach, because one would need to blend two matrices instead of two Vector2.
Offline davedes
« Reply #19 - Posted 2013-11-03 14:46:14 »

@davedes you have to do the rotating or the normals are completely wrong. You might not notice it in this example, but just flip the character on the y-axis and the light is coming from the wrong direction. I mean it is quite common for a ingame char to be able to move from left to right and the other way around.
D'oh... of course.

Though you could just use a uniform for that, so that you are still batching your bones together and without any extra vertex data.

This is of course a single case where the sprite is generally pointing either one or the other direction, and doesn't rotate much on the whole. Other sprites I can see this being a problem. In which case I guess passing more vertex attributes is the best way to deal with it.

Offline pitbuller
« Reply #20 - Posted 2013-11-03 15:05:58 »

About correctly rotated normals. It's not that complicate neither expensive.
You just send angle of bone world rotation(or precalcualted sin/cos from that) and calculate 2x2 matrix at vertex shader and rotate light around z axel with that. If you have multiple lights then its cheaper to just rotate normal. 2d is special case so treat it like one.

SpineSpriteBatch with shader code behind the link http://pastebin.com/8U6vWr0P
Pages: [1]
  ignore  |  Print  
 
 

 

Add your game by posting it in the WIP section,
or publish it in Showcase.

The first screenshot will be displayed as a thumbnail.

xsi3rr4x (15 views)
2014-04-15 18:08:23

BurntPizza (13 views)
2014-04-15 03:46:01

UprightPath (27 views)
2014-04-14 17:39:50

UprightPath (12 views)
2014-04-14 17:35:47

Porlus (29 views)
2014-04-14 15:48:38

tom_mai78101 (51 views)
2014-04-10 04:04:31

BurntPizza (110 views)
2014-04-08 23:06:04

tom_mai78101 (211 views)
2014-04-05 13:34:39

trollwarrior1 (179 views)
2014-04-04 12:06:45

CJLetsGame (185 views)
2014-04-01 02:16:10
List of Learning Resources
by Longarmx
2014-04-08 03:14:44

Good Examples
by matheus23
2014-04-05 13:51:37

Good Examples
by Grunnt
2014-04-03 15:48:46

Good Examples
by Grunnt
2014-04-03 15:48:37

Good Examples
by matheus23
2014-04-01 18:40:51

Good Examples
by matheus23
2014-04-01 18:40:34

Anonymous/Local/Inner class gotchas
by Roquen
2014-03-11 15:22:30

Anonymous/Local/Inner class gotchas
by Roquen
2014-03-11 15:05:20
java-gaming.org is not responsible for the content posted by its members, including references to external websites, and other references that may or may not have a relation with our primarily gaming and game production oriented community. inquiries and complaints can be sent via email to the info‑account of the company managing the website of java‑gaming.org
Powered by MySQL Powered by PHP Powered by SMF 1.1.18 | SMF © 2013, Simple Machines | Managed by Enhanced Four Valid XHTML 1.0! Valid CSS!