Er, unless I'm reading your code wrong you're doing 4096 texture lookups, 10240 branches, 24576 floor() calls and a boat load of math
per pixel.

That's never going to run at a decent speed, and frankly I'm amazed you get 5fps.
I think you're going to have to explain and change your algorithm, because a loop that big and heavy is never going to fly.
Yes, at worst case, it is 4096 lookup. I stop before when I reach the last segment. In the texture, 1 color is 1 float value. I store all segment of the polygon (x1,y1,x2,y2), (...), ...
Then I test if the point is in the polygon. I manage to get 5 fps since I only draw on the bounding of the polygon and not the fullscreen.
All the floor are to remove rounding error when converting the raw data (4 byte) to the orignal data (fixe point) throught float.
The Intel G45 don't support float texture

. Well I have give a try without the floor... no speed change

I'm guessing you're trying to scan-convert your polygons manually in the pixel shader, which makes no sense to me when the graphics card already has highly specialised hardware to do exactly that.
Hardware for antialising filled polygons ? I know none. What I want there is to avoid polygon conversion and triangulator on CPU. And I want to add radial gradiant filling at some point.
Don't point AAA for antialising, G45 don't have it and when I use it on NVidia card it scraw up what I want to do (I don't find how to stop it and restart it in the middle of the draw with LWJGL).
You might want to look at this
I know this. It was the start of my word but :
- it use triangulator
- there is lot of polygon transformation
- it only deals with cubic curve (quadratic one are soooo complicated)
- antialiasing is horrible

. It is nice with low radius curve but not with high radius curve.
When I see people speeking about GPU ray casting voxel, I wonder why it is soo difficult for 2D polygons

How people doing ray casting do ? There is a big loop too, isn't it ?
It is my first really shader so I don't really know what is the limit and what to expect.