AGK - Getting worse over time?

[Qube]

Let begin the great AGK test

For quite a while now I've found the output of AGK getting worse. Especially on Mac.

Since version AGK Classic 2018-07-12 I've found a big drop in raw FPS output and the introduction of judder. As 99% of the time I develop on my Mac I'd be interested to see what other users experience across various OS's and AGK versions.

I've compiled binaries of 3 different versions of AGK for Windows / Mac and Linux. The versions used were :

AGK Classic 2018-07-12
AGK Classic 2019-06-11
AGK Studio 2019-07-23

For me the best performer ( on Mac at least ) for both constant smoothness and fastest FPS ( with vSync off ) is AGK Classic 2018-07-12.

How to test :

1.. Download the version(s) for the OS's you have.
2.. Watch closely for a minute or two to see which AGK version runs the smoothest without juddering.
3.. After point 2 press S to toggle vSync Off and see which version produces the fastest FPS.

Download Links :

https://www.syntaxbomb.com/files/AGK-FPS-Judder-Windows.zip
https://www.syntaxbomb.com/files/AGK-FPS-Judder-Mac.zip
https://www.syntaxbomb.com/files/AGK-FPS-Judder-Linux.zip

Report :

Let me know the results of which worked best / worst for you + The OS's used and the FPS with vSync off per version.

Compile yourself? :

For those that have AGK Classic / Studio / Different versions - Here's the code if you don't want to download the included binaries - Also attached is the ball.png which goes in the media folder.

Code Select Expand


//
// FPS drop and juddering since AGK Classic 2018-07-12
//
// press 'S' to toggle vsync on / off to see speed drop after AGK Classic 2018-07-12
// press ESC to quit this exciting thing
//

SetErrorMode( 2 )
SetWindowSize( 1920, 1080, 1 )
SetVirtualResolution( 1920, 1080 )
SetDisplayAspect( 1920.0 / 1080.0 )
SetOrientationAllowed( 0, 0, 1, 1 )
SetRawMouseVisible( 0 )
SetRandomSeed ( GetUnixTime() )
SetVSync( 1 )
SetClearColor( 0, 0, 0 )
SetDefaultMinFilter( 0 )
SetDefaultMagFilter( 0 )
UseNewDefaultFonts( 1 )

Type tBall
x As Float
y As Float
speed As Float
size As Integer
EndType

Local ball As tBall
Local balls As tBall[]

Local tempImage, ballSprite, vSync, FPS

tempImage = LoadImage( "ball.png" )
ballSprite = CreateSprite( tempImage )
vSync = 1

FPS = CreateText( " FPS : " )
SetTextPosition( FPS, 0, 0 )
SetTextSize( FPS, 30 )

For count = 1 To 200
ball.x = Random( 0, 1919 )
ball.y = Random( 0, 1079 )
ball.speed = Random( 1, 6 ) + ( Random( 1, 50 ) / 100.0 )
ball.size = Random( 32, 32 * 4 )

balls.insert( ball )
Next

Repeat
If GetRawKeyPressed( 83 )
vSync = 1 - vSync

If vSync = 0
SetSyncRate( 5000, 0 )
SetVSync( 0 )
Else
SetVSync( vSync )
EndIf
EndIf

For count = 0 To balls.length
SetSpritePosition( ballSprite, balls[ count ].x, balls[ count].y )
SetSpriteSize( ballSprite, balls[ count ].size, balls[ count ].size )
DrawSprite( ballSprite )

Inc balls[ count ].x, balls[ count ].speed
If balls[ count ].x > 1920 Then balls[ count ].x = 1920 - balls[ count ].x - balls[ count ].size
Next

SetTextString( FPS, " FPS : " + Str( ScreenFPS() ) )
DrawText( FPS )

Swap()
Until GetRawKeyPressed( 27 )


[Amon]

I don't get why they would decline to comment on why this is happening. Is it something they can fix but don't want to because of the focus on AGKS? Or have they borked something in the updates that requires significant work to fix which they are not willing to do due to other technicalities?

I've seen you voice your concerns over there Qube and you may as well speak to the wall. It's dissapointing behaviour on their behalf.

Don't get me started on the absolute sham of a release of AGKS with 90% of what they promissed for release missing.

[plenatus]

@Qube: exact that post you should post on the TGC site.I don´t know about the reaction of TGC but i think its a really good post to ask why this issues doesn´t repaired.

[Derron]

Tested on my linux box:
AGK Classic 2018-07-12:
2000-2100fps, only a bit of jittering here and there - NOT silky smooth

AGK Classic 2019-06-11:
1920-2000fps, started to jitter pretty much after ~15 seconds (jumping by a dozen pixels)
When I've run it a third time - after running all the others - I had no "jumping jitter" for 1 minute (then I stopped)

AGK Studio 2019-07-23:
1950-2050fps, same "tiny bit of jittering" than with "2018-07-12", NOT silky smooth but similary "neglectable"


Watch the "slow and big" balls - they do not move "smooth" when they are slow (but not the slowest).

Interesting: 2018-07-12 was the only version which has shown "marching pixels" on some balls (depends on the speed). Some of the slower had pixels move (just look "top left" of the balls). I was not able to see that in the over versions. It was less visible after I started the other samples and came back to it.
All of this might have to do with: filtering textures and that they place stuff on "pixel coordinates" (so this "subpixelrendering thing" they try to do). Maybe they changed something there?

If stuff "stutters/jitters" it somehow appears as if the affected ball sprite was squeezed (width of 100px becomes 99px) for a short moment.


@ Jumping Jittering
With "jumping jittering" I mean the effect of a ball moving "to the right" and then a bit back "to the left" as if wrong tweening values were passed - or some rounding errors happened.

You might consider adding "left + right" cursorkey support to globally slowdown/speedup movement speed. this "wobbling" (skewing graphics, jumping positions) is only visible for certain speeds (eyes too slow / not tricked). No need to upload the whole thing, just provide an alternative "bytecode.byc". Binaries are just the runtimes - as you know.

bye
Ron

[Steve Elliott]

I tried on Windows (for now)

Version AGK Classic 2018-07-12 worked the best.  It ran smooth and was the only one to correctly run at my native Monitor Refresh of 75hz and 2800FPS with vsync off.

Version AGK Classic 2019-06-11 wasn't smooth, ran at an incorrect vsync of 60 and 2400FPS with vsync off.

AGK Studio 2019-07-23 ran the worst, less smooth, ran at an incorrect vsync of 60 and 1600FPS with vsync off.

[TomToad]

Interesting results here.  On my old system, both classic versions ran smooth at 60 fps.  Vsync off and they both jumped between 300-400 fps.  Did not notice any jittering in either version.
Studio version ran at about 58 fps and jitter was noticeable.  VSync off ran about 180 fps, a huge difference from classic.

Windows 10 Home 64 bit
AMD A8-6410 2.0ghz with AMD Radeon R5 graphics
4 GB Ram

[Qube]

I've posted on the TGC forums so I hope it's seriously looked at and I don't get fanboi fob offs, we'll see.

Studio v1.0 does run the worst for me too but I expect that'll just be brushed away as early Vulkan days. It's clear that there has been a deterioration over both AGK Classic heading into Studio and the longer it goes on the harder it'll be to fix.

[Hezkore]

Specs

• Windows 10
• 144Hz monitor
• RTX 2070 GPU

AGK Classic 2018-07-12

• VSync On = 143-144 FPS. Hard to notice jitter at this FPS, but FPS tends to be a locked 144 in other games.
• VSync Off = ~2000 FPS.

AGK Classic 2019-06-11

• VSync On = 59.9994 FPS. Incorrect VSync. Looks very jittery
• VSync Off = ~1900 FPS.

AGK Studio 2019-07-23

• VSync On = 59.9994. Exactly like version above.
• VSync Off = ~1100 FPS. "Big" drop in FPS.

[plenatus]

@Hezkore:you should post it in the tgc forum.
The vsync limitation issue was corrected some agk versions ago.But it seems its back.

When I see other developers and how they deal with customers, I find it a bit sad at TGC.
Much is ignored and just with Discord you would have the opportunity to actively participate in the development.
I just do not understand.

Maybe its time for an intervention.

[Amon]

We should keep bumping the thread Qube posted to keep it on the first page. Lets prevent it dying out in to the nether pages of that forum. Hopefully they will listen and fix what needs to be fixed.

[Hezkore]

@Hezkore:you should post it in the tgc forum.
The vsync limitation issue was corrected some agk versions ago.But it seems its back.

I've never used AGK for any of my projects, and I don't think I ever will.
So this issue doesn't bother me in any way.
Feel free to use my results though and post on the forum.

I'm only posting my results for Qube.

[Rick Nasher]

Ok, this couldn't resist. ;-)

I too had noticed a drop in performance and smoothness, but thought was due to my old specs. However I now tested on my new laptop(i7-7700HQ@2.80GHz, 8GB, GTX1050, 512GB SSD, W10H V1903 x64) and made some vids to show the initial results.


AGK2 V2018-07-12:
https://i.imgur.com/5zZISK1.mp4
This was by far the best performer(or appeared to be). Average 60fps.


AGK2 V2019-02-08:
https://i.imgur.com/GOkrE04.mp4
Here it really is getting 'the shakes'. Average 59fps.


AGKS V1.0:
https://i.imgur.com/wywEM4T.mp4
Here fps's went down a lot. Average 30fps.
I've tried different compilation settings: VSync. Full speed, 60fps and turns out disabling 'Event based rendering' gives the most smooth movement, but none of the settings up the fps rate significantly.

The later is on Vulkan so should in theory be the faster one I suppose, while above results seem to imply something different. However, it does appear more smooth.



Now.. without VSync enabled the story changes quite a bit:

AGK2 V2018-07-12:
a smooth 265fps.

AGK2 V2019-02-08:
a smooth 255fps.

AGKS V1.0:
a smooth 350fps.


I do not know how to interpret above results correctly though. Any comments on this?

[Derron]

@ videos
I had recorded some with my phone too - but the 25fps recording of a 60fps animation just has to look jerky. No chance to judge fairly based on these videos.


@ laptop fps
Why does your modern laptop limit at <300fps, shouldn't it be a digit more?


@ vsync
I had visible tearing in all 3 binaries. With hardware vsync (rather than a "I ought to refresh every x.yyyy ms") it should not show tearing - except my nvidia driver (Linux) does not play nicely here.
Yet I assume it happens because of this: my main screen (connected via HDMI) is having a refresh rate of "60.02Hz" while the DVI one a refresh rate of "59.95Hz" (according to the nvidia driver panel). The rendering of the tests here happen at a refresh rate of "59.8xxx - 59.9xxxx". Maybe that results in this tearing (as it does not use a hardware vsync).

bye
Ron

[Rick Nasher]

@Derron

@ videos
I had recorded some with my phone too - but the 25fps recording of a 60fps animation just has to look jerky. No chance to judge fairly based on these videos.

Funnily enough the recording doesn't appear to have a visible impact on the jitter or fps at all: it's oddly enough the same with or without recording.
Guess there's some overhead that's reserved for the purpose? I used the Windows in build Game Menu recorder.

@ laptop fps
Why does your modern laptop limit at <300fps, shouldn't it be a digit more?

I'd like to see that indeed. But.. it's not really a gaming laptop. Dunno if this is the reason though.

[Qube]

@Rick Nasher - Cool, thanks for doing that - That pretty much echos my finding on MacOS in regards to the judder too.

I too had noticed a drop in performance and smoothness, but thought was due to my old specs.

Lol, when I first came across it 8+ months ago I thought it was my Mac and was about to reload from fresh but out of curiosity I tried an early version of AGK and that ran just fine.

I do not know how to interpret above results correctly though. Any comments on this?

A GTX 1050 is no slouch and not really much behind my Radeon 580 Pro. I wonder if *drum roll* Vulkan has kicked in with AGK Studio for the raw FPS and that's why it out performs OpenGL of AGK Classic?

Although I find your results very low as when I boot into Windows 10 I get over a 1000+ FPS. Go Mac \o/

@ vsync
I had visible tearing in all 3 binaries. With hardware vsync (rather than a "I ought to refresh every x.yyyy ms") it should not show tearing - except my nvidia driver (Linux) does not play nicely here.
Yet I assume it happens because of this: my main screen (connected via HDMI) is having a refresh rate of "60.02Hz" while the DVI one a refresh rate of "59.95Hz" (according to the nvidia driver panel). The rendering of the tests here happen at a refresh rate of "59.8xxx - 59.9xxxx". Maybe that results in this tearing (as it does not use a hardware vsync).

Do my games that you've played have tearing too?. It's just that all those have delta timing for 60FPS whereas the example code / binaries for this test are just pure sync with no timing so should lock to your monitors refresh.

Gah, this is all getting complex