Texture bleeding - Blitzmax bug with virtual resolution

[Derron]

Ahh my fault ... I use Bruceys maxmods/brl.mod and maxmods/pub.mod

https://github.com/maxmods/brl.mod
https://github.com/maxmods/pub.mod

The updated mods are needed to get a newer "map.mod", sound fixes, AppSuspend-fixes (for linux...) etc.


bye
Ron

[col]

No worries

Building the modules now.
Does the test app scale the sprite sheet to the exact same size as is used in the game?

[Derron]

There is no "scaling" done on the sprites except for "stretching" via DrawSubImageRect(). So yes, it does the same scaling.




bye
Ron

[col]

First off... Note I also DO NOT have the issue on my test unit which uses a Radeon and an Nvidia combo card setup, but I thought I'd check anyway.

As I'm sure you'll appreciate that this kind of thing is time consuming ( but nowhere near as bad as guessing ) but I may have found something that is more than likely of significance.

Some evidential data:


That is a snap of one of that guys youtube vids that clearly shows multiple errors. Other than the obvious pink line there is also a black line to the left of the 'centre left of the 9patch' texture that has bled 1 pixel from the left. I decided to investigate just that one issue for the time being to see if anything shows up.

Tracing through the millions of api calls ( check out the call number at the bottom of the pic - 1842834 ) for frame 292 I found the point at which that texture is used for that centre left part of a 9patch image.
The following image shows the draw calls to render that one section, which I've highlighted in green to the left, to render the part of the image highlighted in red to the right..



Doing the math...
The image is 512 x 512. The UVs for the top left of the texture are 0.423828, 0.0917969 and the bottom right are 0.435547, 0.134766 - this gives texture image coords of 216.999936, 47.0000128 to 223.000064, 69.000192. The section of the texture image that is highlighted in red are pixel positions 217,47 to 223, 68 ( note that this also highlights a plausible reason for the pink lines to the left and bottom of some sprites in the first image ). This particular section of the sprite is then drawn to the currently set framebuffer, at 46,171 with a width of 6 and height 16, ie 40, 171 to 46, 187.

EDIT: The coordinates shown in the window that shows the texture image are from where the mouse is when I took the snap shot - you can verify the real coordinates in the real image used in the game.

You can see a potential issue here with texture coords. Most gpus will sample from the centre of the closest texel and you have... x = 216 in one call and y = 47 in another. So I'd then ask Ok, why does it work on most gpus and only some have the issue... to that I'd hazard a wild guess that it would have something to do with rendering smaller than what you're sampling? I don't know, and my guess is more than likely wildly wrong

Next it would be good to know how texture sampling is setup. You use glTexParameter to do that. Searching back up the call chain ( over 800,000 calls before ) we find this ( which would be the same for all textures used in BlitzMax Max2D ) :



Looking through the OpenGL docs at https://www.khronos.org/registry/OpenGL-Refpages/es2.0/xhtml/glTexParameter.xml  the paramerter of GL_TEXTURE_MAG_FILTER with a value of GL_LINEAR will return a weighted average of the closest 4 pixels to the center of the required pixel.

A quick solution... leave a gap around all of your sprites in the atlas.

Hope it helps

[Derron]

Wow - that is pretty detailled. I hope you had a bit of "fun" crawling through this stuff (read "everything has to be done at least one time in a life").

To keep the reply short: you suggested to leave 1 pixel "free" in all directions.


1) how to handle nine patch then?
Should I always have a "blank line" between the markers and the content? Think this is counter-intuitive and incompatible to the basic design of it on android.


2) how does "weighted average of the closest 4 pixels to the center of the required pixel." help for non-scaled images?
As I understood the docs you linked for "glTexParameter" the 4 pixels are only taken into consideration when doing a "scale". Means it only happens when doing the stretched draw of the "stretchable" sprite portions (eg. center of buttons).

How does that explain the "pink border" on non-scaled sprites (like the overlay-icons of the player and antenna-symbol in the first screen of your post).

But yes, it explains the black border seen in the "chat background" (on the left side) - leading to question 1) again.


3) Doing a "leaving a pixel border blank" leads to "visual offsets"
If you add a blank pixel on top/bottom/left/right of the sprite: only the left, right, top and bottom of a nine patch sprite will "benefit" from it. All other "sub-portions" (like the center) will "blend" into the outer parts when this magic "4 pixels average" is calculated - nothing bad for my GFX but when doing some pixel-art-stuff (fine pixel-curves) it will break things visually.

Also it is an issue if the unstretched part is using the original color while the stretched one mixes in the blank lines (I assume so at least) which leads to shallow/faded-out colors (1.0*srcRGBA + 1.0*blank)/2.

Wouldn't it be better to disable the used TexParameter so it does the same on all computers this specific app is run on? like disabling "smoothing on scaling" for certain textures.


----
Edit: Might it help to disable "mipmapping" for these images? Eg. the GUI-spritesheet is loaded with "FILTEREDIMAGE | MIPMAPPED" (via "config/gui.xml").


Stated there: https://gamedev.stackexchange.com/questions/46963/how-to-avoid-texture-bleeding-in-a-texture-atlas
One user wrote that mipmap is not useful for 2D - and suggested to offset the pixels on the texture by "0.5" - he calls it "half pixel correction".


Thoughts on this?
----

Again: big thanks for the time you already spend on "digging".

bye
Ron

[col]

Yeah I found it fun  I don't mind when you can use the tools to find the problem hehe.
GpuPerfStudio actually kept crashing so I ended up using ApiTrace - super invaluable tools for gpu stuff !!! They are as essential to debugging gpus as a regular debugger is to debugging your cpu code.

As I say I use RenderDoc and it's way more detailed than the app that I used here.

For the solution.. hehe that's your job!! I just helped out with finding the problem 

Kidding.
You could try disabling mip maps and also setting the value to GL_NEAREST to see if it works? It may make it worse in which case fixing the UVs would then be best.

[col]

The 0.5 offset is a misnomer. There's something wrong with the viewport, render target size, view projection matrix, pixel position math - any or all of them - if you need the 0.5 offset. You should be rendering at X to X + width in pixels to be accurate. If the calculation both sides of the image are coming from a scale then you can end up with wrong values due to rounding errors.

By all means try the 0.5 offset - I mean if it works then all is good yes?

[Derron]

I will ask the user to remove that filter-lines from the XML so the image is loaded "non-mipmapped". But imho this should only solve "bleeding" (like faded out borders). Cannot imagine how this introduces the "black lines" (of the ninepatch sprites) when drawing the sprite unscaled (so no "gl_linear" should be ever be of interest).

Edit: I saw some "errors" (in that video) in sprites drawn in the mainscreen too (talking about plants in the building - or the human beings on the bottom right of the interface). They are already loaded without "mipmapping" (FILTEREDIMAGE,MASKEDIMAGE,DYNAMICIMAGE for the plants, and "0" for the humans). So especially for the "humans" it is just drawing them "plain" without stretching or scaling - and still they get this pink border (In that case I know that a "blank border" will help to get rid of the line


But most important is: How does it explain the "offset" (not talking about bleeding - which would result eg. in a mix of "pink border + actual pixel color") ?




@ Tools
I already did not get over with Valgrind and the likes. Glad to get "gdb" running so I can hand over some backtrace to Brucey if things are borked up with the NG compiler results :p
Digging in "logs" is something I do not like at all - last time I did that was for analyzing some wireshark traffic logs.




bye
Ron

[col]

How does that explain the "pink border" on non-scaled sprites

If you look at the UV coords in the samples gpu calls they are 1 off to the left and 1 down.

Yeah digging through logs is boring. These are visual tools so it's a bit easier to see whats going on.

[Derron]

I edited my previous post meanwhile, so you might have missed the "edit:" part.


@ 1 pixel offset
But where does it come from? And if you already saw this offset in your run, why did it not lead to artifacts/bleeding on your computer? I am talking about sprites which end right before a pink border, not sprites with a bit of blank space until a pink border comes.

If there was a mistake in the calculation of sub-image-coordinates I thought of seeing it on my computer too.




Edit:

Doing the math...
The image is 512 x 512. The UVs for the top left of the texture are 0.423828, 0.0917969 and the bottom right are 0.435547, 0.134766 - this gives texture image coords of 216.999936, 47.0000128 to 223.000064, 69.000192. The section of the texture image that is highlighted in red are pixel positions 217,47 to 223, 68 ( note that this also highlights a plausible reason for the pink lines to the left and bottom of some sprites in the first image ). This particular section of the sprite is then drawn to the currently set framebuffer, at 46,171 with a width of 6 and height 16, ie 40, 171 to 46, 187.

If this is the reason for the offset I seem to not have understood that paragraph correctly. What is wrong with the calculated values (when rounding them to integers, not floats)? It seems to correctly describe the portion on the spritesheet (especially when seeing your rect-rectangle - it is not offset or so).

bye
Ron

[col]

If there was a mistake in the calculation of sub-image-coordinates I thought of seeing it on my computer too.

I know, I know, I would have thought that I'd see it ad well. I remember that I had a very similar issue with the d3d11 driver once - on some, but not all, ati cards the image would be 1 pixel off in both directions. It didn't affect Nvidia cards at all. Who knows what tricks go on in the circuitry to make it all look ok. But at the end of the day... the numbers are there.

I used ApiTrace from https://apitrace.github.io/. It may be worth you downloading and taking a look for yourself - its very easy to use.

[col]

What is wrong with the calculated values (when rounding them to integers, not floats)?

Gpus will round to the nearest centre, ie the nearest 0.5, not to the nearest 0.0.

216.999936 != 217.5          - gpu will use 216.5, 1 pixel to the left
47.0000128 == 47.5
223.000064 == 223.5
69.000192 != 68.5          - gpu will use 69.5, 1 pixel down

You intend to render the pixels from
217,47 to 223, 68

but using the UVs given to it the gpu will use coords of
216.5, 47.5 to 223.5, 69.5

ie 216, 47 to 223, 69.

Hence the reason people add 0.5 to the coords.. to fix a kludge somewhere else. Gpus already do the 0.5 thing internally.

[Derron]

Ah, so the "216.9999936" should normally be already something "near" 217.5 or so?


Already installed the tool - clicking on something always needs a "replay" until it reaches that point in the trace. A bit tedius to find the right spot :-)



Shouldn't the "sample" code (doing just a button + drawing the spritesheet) be faster to debug?
Think I need to catch some sleep before even trying to fix that as I am not knowing the "correct value" now - even if you wrote that already in the previous posts...
(But I guess i will still try instead of sleeping)




bye
Ron

[col]

Already installed the tool

Good man

Yes, they all replay but that one replays quite slow, maybe for accuracy - and that's more important. Others can be very fast in replaying.

As I said before it can be slow work and it involves manual math too  it's definitively a process that you don't want to rush and when you set everything up correctly you can be confident that it will work correctly on everyones pc.

I'd catch some zzz's and do it tomorrow with a fresh mind and fresh perspective

[Derron]

Ok, then a last question until I will have a look on how my pillow feels :p

The UVs for the top left of the texture are 0.423828, 0.0917969 and the bottom right are 0.435547, 0.134766 - this gives texture image coords of 216.999936, 47.0000128 to 223.000064, 69.000192

Ok, so the coords "216.9999" are incorrect as they _may_ lead to use 216.5 instead of 217.5 ?
If so - the calculation of the subimage-coordinate is resulting in ~217:

This is what I pass (using a "print-debug" in my sample file just drawing the input-widgets "left border"):
DrawSubImageRect(parent.image, 70.0000000, 26.0000000, 6.00000000, 16.0000000, 217.000000, 47.0000000, 6.00000000, 22.0000000)

So the "217" is there. I assume I am not supposed to manually add the 0.5 here - but couldn't this help (similar to the "int(float + 0.5)" rounding mechanism) ?
I mean, if I add 0.5 to 217 it might result in either "216.99999 + 0.49999 = 217.499998" or "217.00001 + 0.500001 = 217.500002" which is in both cases "near" 217.5 instead of 216.5
BUT ... if that was the solution to use, shouldn't  this correction be in max2D (drawsubimagerect()) rather than my code?
Ok, so I added that 0.5 to the sourceX and sourceY and ... results were weird as I then saw black borders in other portions (of course, as I now "added" the right-side-ninepatch-borders as it seems that my GPU "rounds up" instead of "down" as the let's players gpu does).
Adding "0.1" resulted in a correct display here. Might it be that the users GPU/driver (a nvidia 650GT if I remember correctly) truncates these values - so eg. "216.9999" becomes 216 - and so it uses "216.5"? In that case the "216.99999 + 0.1" leads to "217" and then "217.5". On our "normal" computer setups it should still work as "0.1" does not round to the next texel, I think.



Thanks a lot for your assistance.

bye
Ron