May 24, 2019, 11:18:25 PM

Author Topic: Need Help Optimizing Particle System  (Read 285 times)

Offline Ashmoor

  • Jr. Member
  • **
  • Posts: 26
Need Help Optimizing Particle System
« on: April 29, 2019, 01:54:21 AM »
I am working on a 2d particle system in BlitzMax 1.50 and it seems like my computer can only handle about 800 particles before slowing down. Given that I run an i7 at 4Ghz, I would guess I'm doing something wrong. I would like to optimize the code but I have no idea how or where to start. I also noticed that different particle types load the system differently.

I am using fixed rate logic, running at 100 updates/second and drawing at only 60. I create particles using an emitter and each particle is stored by the particle system in a TList. My particle type has ~85 fields and can rotate, fade, move, frame animate, scale and interact with point attractors (like gravity) based on mass, has basic air drag, etc.

I am using virtual resolution, scaling down from 1920x1080(full screen) to 1366x768

Any hint about what could be slowing down my program or how to go about finding the bottle neck would be appreciated.

Offline Qube

  • Administrator
  • Hero Member
  • *****
  • Posts: 1828
Re: Need Help Optimizing Particle System
« Reply #1 on: April 29, 2019, 04:16:16 AM »
85 fields for a particle system? - That seems excessive :o

It shouldn't be choking at 800 particles so clearly there is something wrong. Without seeing your code it's hard to say what is wrong but basic big gotcha's are usually :

1.. Using lots of string compares in a big loop. This is a frame rate killer.
2.. Sorting a lot of data every frame, also slow.
3.. Recursion on a routine. Again very slow per frame when it's constantly calling itself.

Also if you're checking 85 fields per frame can be a frame rate killer, especially of there are strings involved.

Really we need to see your code to figure out where the issue is but 800 particles should not be an issue.
Until the next time...

Offline Yellownakji

  • Full Member
  • ***
  • Posts: 216
  • Whoop-de-doo-ba-dooo!
Re: Need Help Optimizing Particle System
« Reply #2 on: April 29, 2019, 04:19:04 AM »
....

Some questions:

1. Are you using threads? (not 'build threaded' but utilizing ':Tthread' and 'WaitThread()'?
2. 'Fixed Rate' logic?  Have you tried to use a delta timer?  Delta is the most accurate for FPS capping / keeping the game steady under load.
3. Do you *need 800 particles?  Annoying question, i know.  But i'm asking anyways.  This is a forum, after all.
4. Exactly what kind of calls are you making per frame?

Offline Derron

  • Hero Member
  • *****
  • Posts: 2087
Re: Need Help Optimizing Particle System
« Reply #3 on: April 29, 2019, 07:16:47 AM »
@ Yellownakji
Fixed Rate Logic can still use deltas when rendering. It fixes the rate of the _logic_ not of the rendering. And with the "delta" being constant you then have to take care of the "inbetweens" by using tweening.


@ particle performance
You say you have a big bunch of properties.
Are you handling them individual in "UpdatePropertyXYZ()" calls or at least "split" into logic blocks? Disable a pile of the property-logic and check how big the numbers of max particles grow. If even without "physics/logic" your number is pretty low then the general logic (tlist + handling adding/removing/iterating, ...) has flaws.
If numbers grow in a pretty good fashion then enable some properties until you see dropdowns in the max particle value.

Generic hints:
String comparison can become slow so Brucey once wrote "TLowerString" for me:
https://github.com/GWRon/Dig/blob/master/external/string_comp.bmx
https://github.com/GWRon/Dig/blob/master/external/string_comp.c
It is some kind of "fast string comparator" (as long as you do not need case-sensitive comparison as it - as the name says - does a "lower string" comparison). I use it often as eg. my GUI system utilizes strings for "layers" (or "groups") and this allows a nearly unlimited combination and filtering ("myscene.system.layer0"). Same stuff I use for assets (GetSpriteFromRegistry("gui.button.normal.down")).


Next to strings "complex" mathematic operations can create slowdowns here and there:
- sin/cos/tan
- sqrt
- "x^20"
- ...


bye
Ron

Offline Ashmoor

  • Jr. Member
  • **
  • Posts: 26
Re: Need Help Optimizing Particle System
« Reply #4 on: April 29, 2019, 12:12:38 PM »
@Qube
"85 fields for a particle system? - That seems excessive"

I have no idea how to do it otherwise.


@Yellownakji
"Are you using threads? (not 'build threaded' but utilizing ':Tthread' and 'WaitThread()'?"

No, I don't know how to do that.

"'Fixed Rate' logic?  Have you tried to use a delta timer?  Delta is the most accurate for FPS capping / keeping the game steady under load."

I thought Fixed Rate is the most steady because it can drop to 20 updates per second and still draw at 40+

"Do you *need 800 particles?  Annoying question, i know.  But i'm asking anyways.  This is a forum, after all."

I think I do. I have scenes with 2-3 waterfalls, fire torches and smoke + flower petals falling from trees.

"Exactly what kind of calls are you making per frame?"

I tried different calls, usually I update stuff like position, scale, angle and alpha. There are no collisions. Before drawing I update the linear interpolation of position, scale etc.


@Derron
"Are you handling them individual in "UpdatePropertyXYZ()" calls or at least "split" into logic blocks?"

My update function has calls to Move() Rotate() Scale() etc. All of which start with a check like this "if not isMoving then return"

"If numbers grow in a pretty good fashion then enable some properties until you see dropdowns in the max particle value."

I'll try to do that.

Here is my particle type:
Code: [Select]

Type TParticle

Field shapeType:Int = 1 '1 - circle; 2 - rect; 3 - 2 circles and line

Field xPos:Float
Field yPos:Float
Field zPos:Int

Field image:TImage
Field blendMode:Int = 0 '0 is normal alpha blend, 1 is lightblend

Field id:Int
Field idName:String

Field dragSize:Float = 1
Field dragSizeRndAdd:Float = 0

'coloring parameters
Field isColored:Float = 0 'this states weather or not to apply a color blend
Field clrR:Int = 255
Field clrG:Int = 255
Field clrB:Int = 255


Field lastPos:TVector2d = New TVector2d 'last pos coords
Field curPos:TVector2d = New TVector2d 'cur pos coords
Field velocity:TVector2d = New TVector2d
Field acceleration:TVector2d = New TVector2d

Field mass:Float = 1.0
Field massRndAdd:Float

Field qbTrajectory:TQuadraticBezier 'make sure path is parametrized by arc len
Field isOnPath:Int = False
Field posOnPath:Int = 0 'position in the path array of coords

Field isComet:Int = False
Field cometTrailSpawnDelay:Int = 100 'ms
Field cometTrailSTL:Int 'comet trail spawn time left
'Field cometTrailType:int 'this is in case I need more than one comet trail type

Field timeToLive:Int 'actual time to live in ms - lifetime
Field timeToLiveBase:Int 'time to live base in ms
Field timeToLiveRndAdd:Int = 0 'extra time range to add to base in ms
Field isDead:Int = False


'there should be a scaleType, fadeType which would decide
'on various scaling functions instead of just the liniar ones
'

'------------- SCALE params
Field isScaling:Int = 0 'scale switch

Field curScale:Float = 1.0
Field lastScale:Float = 1.0
Field drawScale:Float = 1.0 'variable used for setting the part scale before drawing
Field scaleOrig:Float = 1.0 'scale to start at
Field scaleDest:Float 'max scale to reach
Field scaleSpeed:Float 'scale speed used for computing scaling
Field scaleSpeedBase:Float 'base used for scale speed
Field scaleSpeedRndAdd:Float 'random upper bound for scale spd
Field scaleSquish:Float = 1 'this is the report of yScale/xScale

'------------- FADE params
Field isFading:Int = 0 'fade switch
Field fadingType:Int = 1 '1 - standard fade vrom alpha orig to alpha dest, 2 - oscilator; 3 - att oscilation from orig to dest

Field fadeOscPeriod:Float
Field fadeOscPeriodRndAdd:Float
Field curFadeOscFrame:Float = 1.0

Field curAlpha:Float = 1.0
Field lastAlpha:Float = 1.0
Field drawAlpha:Float = 1.0 'variable used for drawing

Field alphaOrig:Float = 1.0 'initial transparency
Field alphaDest:Float 'destination transparency
Field fadeSpeed:Float '
Field fadeSpeedBase:Float
Field fadeSpeedRndAdd:Float


'------------- ROTATION params
Field isRotating:Int = 0
Field drawAngle:Float = 0.0 'the actual angle used for drawing
Field curAngle:Float = 0.0
Field lastAngle:Float = 0.0

Field angularVelocity:Float = 0.0 'actual sim angular velocity
Field angularAcceleration:Float = 0.0 'actual sim angular accel

Field angularVelocityBase:Float = 0.0 'initial angular velocity - used for preset liniar rotations
Field angularVelocityRndAdd:Float = 0.0 'initial random to add to the base
Field angularAccelerationBase:Float = 0.0 'initial angular accel - used for preset liniar accels
Field angularAccelerationRndAdd:Float = 0.0 'initial random to add to the base
Field angularVelocityCap:Float = -1 'this is the maximum angular velocity, infinite if -1
Field rotationDirection:Int = 1 '1 cw, 0 rnd, 2 ccw

'------------- ANIMATION params
Field isAnimating:Int = 0
Field drawFrame:Float
Field curFrame:Float
Field lastFrame:Float

Field animSpeed:Float 'this should be in frames per second
Field animSpeedRndAdd:Float = 0.0
Field animType:Int '0 - static; 1 - forward loop; 2 - bounce; 3 - choose random frame
Field animDirection:Int '1- forward; (-1) - backwards; 0 is random

Field useDrag:Int = True
Field useFriction:Int = True
Field useWind:Int = True
Field useAttractor:Int = True
Field useGravFall:Int = True
Field gravFallMultiplier:Float = 1.0 'this is used for tweaking individual particles

Field attractorId:Int = -1
Field attractorGroupId:Int = -1

Field target_xPos:Float 'target x coord
Field target_yPos:Float 'target y coord
Field hasTarget:Int = False 'target exists
Field targetRange:Float = 50 'range to act on target hit


Method UpdateSelf()

Move()
Scale()
Fade()
Rotate()
Animate()

If isComet Then UpdateComet()

'countdown to extinction
timeToLive:-(1000 / (1 / frlManager.dt))
If timeToLive < 0 Then
isDead = True
EndIf

'die on arrival at destination
If hasTarget Then
If CheckTargetHit() Then isDead = True
EndIf

End Method

'Summary: updates stuff related to comet properties
Method UpdateComet()
'countdown to spawn
cometTrailSTL:-(1000 / (1 / frlManager.dt))
If cometTrailSTL <= 0 Then
'spawn trail
vfxPEmitter.EmitCometTrail(curPos.dx, curPos.dy)
cometTrailSTL = cometTrailSpawnDelay
EndIf
End Method


Method CheckTargetHit:Int()
Local th:Int = False

If (target_xPos - curPos.dx) ^ 2 + (target_yPos - curPos.dy) ^ 2 < targetRange ^ 2 Then th = True

Return th
End Method

'Summary: update drawing parameters from sim parameters
Method UpdateFRLDraw(alpha:Double)
'update pos
xPos = curPos.dx * alpha + lastPos.dx * (1 - alpha)
yPos = curPos.dy * alpha + lastPos.dy * (1 - alpha)

'update scale
drawScale = curScale * alpha + lastScale * (1 - alpha)

'update transparency
drawAlpha = curAlpha * alpha + lastAlpha * (1 - alpha)

'update rotation
drawAngle = curAngle * alpha + lastAngle * (1 - alpha)

'update animation
drawFrame = curFrame * alpha + lastFrame * (1 - alpha)
End Method



Method DrawSelf()
SetAlpha(drawAlpha)
SetScale(drawScale, drawScale * scaleSquish)
SetRotation(drawAngle)
If isColored Then SetColor(clrR, clrG, clrB)
If blendMode = 1 Then SetBlend(LIGHTBLEND)
If Len(image.frames) > 1 Then
DrawImage(image, xPos, yPos, drawFrame)
Else
DrawImage(image, xPos, yPos)
EndIf
SetBlend (ALPHABLEND)
SetColor(255, 255, 255)
SetRotation(0)
SetScale(1.0, 1.0)
SetAlpha(1.0)

'DrawDebug()
End Method

Method DrawDebug()

Local size:Float = mass * 10
Local half:Float = size / 2

Select shapeType
Case 1
DrawOval(xPos - half, yPos - half, size, size)
Case 3
SetRotation(drawAngle)
DrawRect(xPos - 20, yPos - 20, 40, 40)
'DrawOval (xPos - 33, yPos - 3, 6, 6)
'DrawOval (xPos + 33, yPos - 3, 6, 6)
'DrawLine (xPos - 30, yPos, xPos + 30, yPos)
SetRotation(0)
End Select


velocity.DrawSelf(xPos, yPos, 1 / frlManager.dt)
Local tx:String = GetTwoDecimalFloat(velocity.magnitude)
tx = tx + "; " + GetTwoDecimalFloat(mass)
'Local tx:String = GetTwoDecimalFloat(velocity.magnitude)
If drawDebugInfo_switch Then DrawText(tx, xPos - 50, yPos - 50)
End Method

'-------------------------------------------------- MOVEMENT
'Summary: apply force to move the object
Method ApplyForce(force:TVector2d)
Local f:TVector2d = TVector2d.Divide(force, mass)
acceleration.AddVector(f)
EndMethod

Method Move()
'update motion if needed
If isOnPath Then
'advance on path
If posOnPath + 1 < Len(qbTrajectory.arcLenPointsArr) Then
posOnPath:+1
'If qbTrajectory.arcLenPointsArr[posOnPath] Then
lastPos.CopyVector(curPos)
curPos.SetPos(qbTrajectory.arcLenPointsArr[posOnPath].x, qbTrajectory.arcLenPointsArr[posOnPath].y)
Else
isDead = True
ActOnCometReachDestination(Self)
'DebugStop()
EndIf

Return
EndIf

lastPos.CopyVector(curPos)

velocity.AddVector(acceleration)
'If velocity.magnitude > velocity.limit Then velocity.ClipMagnitude()
curPos.AddVector(velocity)
acceleration.MultiplyByScalar(0)

EndMethod

'Summary: Stops the particle motion.
Method StandStill()
acceleration.MultiplyByScalar(0)
velocity.MultiplyByScalar(0)
End Method

'Summary: Instantly sets particle position
Method SetPos(xIni:Float, yIni:Float)
xPos = xIni
yPos = yIni
curPos.SetPos(xPos, yPos)

lastPos.CopyVector(curPos)
End Method

'--------------------------------------------------- SCALING
'Summary: Computes particle scaling
Method Scale()
If Not isScaling Then Return
lastScale = curScale
curScale:+scaleSpeed

If scaleSpeed > 0 Then
If curScale > scaleDest Then SetPartScale(scaleDest) ; isScaling = False
EndIf

If scaleSpeed < 0 Then
If curScale < scaleDest Then SetPartScale(scaleDest) ; isScaling = False
EndIf

End Method

'Summary: Insta set part scale
Method SetPartScale(sc:Float)
lastScale = sc
curScale = sc
drawScale = sc
End Method


'--------------------------------------------------- FADING
'Summary: Compute particle fading
Method Fade()
If Not isFading Then Return
lastAlpha = curAlpha

Select fadingType
Case 1
curAlpha:+fadeSpeed
If Abs(alphaDest) - Sgn(fadeSpeed) * (Abs(curAlpha)) < 0 Then
SetPartAlpha(alphaDest)
isFading = False
EndIf

Case 2 'oscilation between orig and dest
curFadeOscFrame:+1
If curFadeOscFrame > fadeOscPeriod Then curFadeOscFrame = 1

Local pathPos:Float = curFadeOscFrame / fadeOscPeriod
Local alphaCen:Float = (alphaOrig - alphaDest) / 2
Local amplitude:Float = alphaOrig - alphaDest
Local ang:Float = 360 * pathPos
curAlpha = alphaOrig - Abs(amplitude * Cos(ang))

End Select




EndMethod

'Summary: Insta set part alpha
Method SetPartAlpha(alp:Float)
curAlpha = alp
lastAlpha = alp
drawAlpha = alp
End Method


'--------------------------------------------------- ROTATING
'Summary: Compute particle rotation
Method Rotate()
If Not isRotating Then Return
lastAngle = curAngle

If angularVelocityCap <> - 1 Then
If Abs(angularVelocity) < Abs(angularVelocityCap) Then
angularVelocity:+angularAcceleration
Else
angularVelocity = angularVelocityCap
EndIf
Else
angularVelocity:+angularAcceleration
EndIf

curAngle:+angularVelocity
End Method

'Summary: Insta set part angle
Method SetPartRotation(ang:Float)
lastAngle = ang
curAngle = ang
drawAngle = ang
End Method


'--------------------------------------------------- ANIMATING
Method Animate()
If Not isAnimating Then Return

lastFrame = curFrame
curFrame:+animSpeed

Select animType

Case 1 'fw or bk loop
If curFrame > Len(image.frames) - 1 And animSpeed > 0 Then curFrame = 0
If curFrame < 0 And animSpeed < 0 Then curFrame = Len(image.frames) - 1

Case 2 'bounce
'do nothing yet
'add bounce code here if needed

End Select

End Method



'Summary: Insta set part frame
Method SetPartFrame(fr:Float)
lastFrame = fr
curFrame = fr
drawFrame = fr
End Method





'Summary: Intializes the particle based on it's parameters
Method initPart()

'init life
timeToLive = timeToLiveBase + Rand(0, timeToLiveRndAdd)

dragSize = Rnd(dragSize, dragSize + dragSizeRndAdd)
mass = Rnd(mass, mass + massRndAdd)

'init scaling
If isScaling Then
'these should be adjusted so that they stay consistent no matter the UPS rate
scaleSpeed = Sgn(scaleDest - scaleOrig) * (scaleSpeedBase + Rnd(0, scaleSpeedRndAdd))
EndIf

SetPartScale(scaleOrig)

'init fading
If isFading Then
Select fadingType
Case 1
'these should be adjusted so that they stay consistent no matter the UPS rate
fadeSpeed = Sgn(alphaDest - alphaOrig) * (fadeSpeedBase + Rnd(0, fadeSpeedRndAdd))
SetPartAlpha(alphaOrig)
Case 2
fadeOscPeriod = Rnd(fadeOscPeriod, fadeOscPeriod + fadeOscPeriodRndAdd) * (1 / frlManager.dt) 'sec * ups
SetPartAlpha(0.0)
EndSelect
Else
SetPartAlpha(alphaOrig)
EndIf

If isRotating Then
'these should be adjusted so that they stay consistent no matter the UPS rate
Local rotDir:Int
If rotationDirection <> 0 Then rotDir = rotationDirection Else rotDir = Sgn(Rnd(-2, 2))
angularVelocity = Sgn(rotDir) * (Abs(angularVelocityBase) + Rnd(0, angularVelocityRndAdd))
angularAcceleration = Sgn(angularAccelerationBase) * (Abs(angularAcceleration) + Rnd(0, angularAccelerationRndAdd))
EndIf

SetPartRotation(drawAngle)

If animType = 3 Then
SetPartFrame(Rnd(0, Len(image.frames) - 1))
EndIf

If animDirection = 0 Then animDirection = Sgn(Rnd(-1, 1))
animSpeed = animDirection * Rnd(animSpeed, animSpeed + animSpeedRndAdd) / (1 / frlManager.dt) 'convert to fps

End Method






'----------------------------------------------------------------
'             H E L P E R   F U N C T I O N S
'----------------------------------------------------------------
'use this function for quick and dirty particle creation for testing
Function CreateTParticle:TParticle(xIni:Float, yIni:Float, mIni:Float)
Local part:TParticle = New TParticle
part.SetPos(xIni, yIni)
part.mass = mIni

Return part
End Function

'this is the function that will pass on relevant data to the emitter
Function CopyParticleProperties:TParticle(part:TParticle)
Local p:TParticle = New TParticle
p.SetPos(part.xPos, part.yPos)

p.id = part.id

'image related parameters
p.image = part.image
p.blendMode = part.blendMode
p.isColored = part.isColored
p.clrR = part.clrR
p.clrG = part.clrG
p.clrB = part.clrB

'forces
p.dragSize = part.dragSize
p.dragSizeRndAdd = part.dragSizeRndAdd

p.mass = part.mass
p.massRndAdd = part.massRndAdd

p.useDrag = part.useDrag
p.useGravFall = part.useGravFall
p.gravFallMultiplier = part.gravFallMultiplier
p.useWind = part.useWind
p.useAttractor = part.useAttractor

'lifetime params
p.timeToLiveBase = part.timeToLiveBase
p.timeToLiveRndAdd = part.timeToLiveRndAdd

'scaling params
p.isScaling = part.isScaling
p.scaleOrig = part.scaleOrig
p.scaleDest = part.scaleDest
p.scaleSpeedBase = part.scaleSpeedBase
p.scaleSpeedRndAdd = part.scaleSpeedRndAdd
p.scaleSquish = part.scaleSquish

'fading params
p.isFading = part.isFading
p.fadingType = part.fadingType
p.alphaOrig = part.alphaOrig
p.alphaDest = part.alphaDest
p.fadeSpeedBase = part.fadeSpeedBase
p.fadeSpeedRndAdd = part.fadeSpeedRndAdd
p.fadeOscPeriod = part.fadeOscPeriod
p.fadeOscPeriodRndAdd = part.fadeOscPeriodRndAdd

'rotating params
p.isRotating = part.isRotating
p.angularVelocityBase = part.angularVelocityBase
p.angularVelocityRndAdd = part.angularVelocityRndAdd
p.angularVelocityCap = part.angularVelocityCap

p.angularAccelerationBase = part.angularAccelerationBase
p.angularAccelerationRndAdd = part.angularAccelerationRndAdd
p.rotationDirection = part.rotationDirection

'animation params
p.isAnimating = part.isAnimating
p.animType = part.animType
p.animSpeed = part.animSpeed
p.animSpeedRndAdd = part.animSpeedRndAdd
p.animDirection = part.animDirection
p.drawAngle = part.drawAngle

p.attractorGroupId = part.attractorGroupId
p.attractorId = part.attractorId

p.target_xPos = part.target_xPos
p.target_yPos = part.target_yPos
p.hasTarget = part.hasTarget

p.isOnPath = part.isOnPath
p.posOnPath = part.posOnPath
If p.isOnPath Then
p.qbTrajectory = New TQuadraticBezier
'p.qbTrajectory.CopyQBezierTrajectory(part.qbTrajectory)
p.qbTrajectory.CopyQBezier(part.qbTrajectory)
EndIf

p.isComet = part.isComet
p.cometTrailSpawnDelay = part.cometTrailSpawnDelay
p.idName = part.idName
Return p
End Function


EndType

And here is my Particle System type - this manages particles and applies various forces to sets of particles.

Code: [Select]

Type TPartSystem

'---------------------------------------------- FORCES DEFINITIONS
'friction
Field frictionCoef:Float = 0.05
Field frictionNormal:Float = 1
Field frictionMag:Float = frictionCoef * frictionNormal
Field friction:TVector2d = New TVector2d

'Gravity
Field gravityAccel:Float = 0.98 * 0.02 'gravitational accel of Earth
Field gravityBigG:Float = 6.67428 'gravitational constant
Field gravity:TVector2d = TVector2d.CreateVecByDirMag(-90, gravityAccel)

'Wind
Field windForce:Float = 0.03
Field wind:TVector2d = New TVector2d

'Drag
Field dragCoef:Float = 0.009 'if this is too high the object will just be stationary
Field drag:TVector2d = New TVector2d

Field id:Int

Field useDrag:Int = 0
Field useFriction:Int = 0
Field useWind:Int = 0
Field useAttractor:Int = 0
Field useGravFall:Int = 0
Field useMutualAttr:Int = 0
Field useMutualRepel:Int = 0

Field entityList:TList = New TList

Field isActive:Int = True

Method UpdateSelf()

If Not isActive Then Return

'update emitters
For Local e:TPartEmitter = EachIn entityList
e.UpdateSelf()
Next

'update particles
For Local p:TParticle = EachIn entityList
p.UpdateSelf()

If Self.useGravFall And p.useGravFall Then ApplyGravFall(p)
If Self.useDrag And p.useDrag Then ApplyDrag(p)
If Self.useWind And p.useWind Then ApplyWind(p)
If p.attractorGroupId <> - 1 Then ApplyGroupAttractor(p)
If p.attractorId <> - 1 Then ApplySingleAttractor(p)
totalPartCount:+1
If p.isDead Then ListRemove(entityList, p)
Next

For Local a:TAttractor = EachIn entityList
a.Updateself()
If a.isDead Then entityList.Remove(a)
Next

End Method

Method UpdateFRLDraw(alpha:Double)
'update emitters
For Local e:TPartEmitter = EachIn entityList
e.UpdateFRLDraw(alpha)
Next

'update particles
For Local p:TParticle = EachIn entityList
p.UpdateFRLDraw(alpha)
Next

For Local a:TAttractor = EachIn entityList
a.UpdateFRLDraw(alpha)
Next

End Method

Method DrawSelf()
'draw emitters
For Local e:TPartEmitter = EachIn entityList
e.DrawSelf()
Next

'draw particles
For Local p:TParticle = EachIn entityList
p.DrawSelf()
Next

For Local a:TAttractor = EachIn entityList
a.DrawSelf()
Next

End Method

Method DrawSelfByZ(curZ:Int)
'draw emitters
For Local e:TPartEmitter = EachIn entityList
If e.zPos = curZ Then e.DrawSelf()
Next

'draw particles
For Local p:TParticle = EachIn entityList
If p.zPos = curZ Then p.DrawSelf()
Next
End Method

'Summary: Draws everything up to curZ
Method DrawSelfUpToZ(curZ:Int)
'draw emitters
For Local e:TPartEmitter = EachIn entityList
If e.zPos <= curZ Then e.DrawSelf()
Next

'draw particles
For Local p:TParticle = EachIn entityList
If p.zPos <= curZ Then p.DrawSelf()
Next
EndMethod

'Summary: Draws everything with z higher than curZ
Method DrawSelfHigherThanZ(curZ:Int)
For Local e:TPartEmitter = EachIn entityList
If e.zPos > curZ Then e.DrawSelf()
Next

'draw particles
For Local p:TParticle = EachIn entityList
If p.zPos > curZ Then p.DrawSelf()
Next
End Method

'Summary: apply downwards grav to particle
Method ApplyGravFall(part:TParticle)
Local tmpGrav:TVector2d = New TVector2d
tmpGrav.CopyVector(Self.gravity)
tmpGrav.MultiplyByScalar(part.mass)
tmpGrav.MultiplyByScalar(part.gravFallMultiplier)
part.ApplyForce(tmpGrav)
End Method

Method ApplyDrag(part:TParticle)
'drag= velocity mag ^2 * drag coef * inverse velocity normalized vector
Local tmpDrag:TVector2d = New TVector2d
tmpDrag.CopyVector(part.velocity)
tmpDrag.MultiplyByScalar(-1)
tmpDrag.NormalizeSelf()
tmpDrag.MultiplyByScalar(Self.dragCoef * part.velocity.magnitude ^ 2 * part.dragSize)
part.ApplyForce(tmpDrag)
End Method

Method ApplyWind(part:TParticle)
If wind.magnitude <> 0 Then part.ApplyForce(wind)
End Method

Method ApplyGroupAttractor(part:TParticle)
For Local a:TAttractor = EachIn entityList
If a.groupId = part.attractorGroupId Then part.ApplyForce(a.CalculateAttraction(part))
Next
End Method

Method ApplySingleAttractor(part:TParticle)
For Local a:TAttractor = EachIn entityList
If a.id = part.attractorId Then part.ApplyForce(a.CalculateAttraction(part))
Next

End Method

Method ActivateSelf()
isActive = True
End Method




Function CreateParticleSystem:TPartSystem()
Local ps:TPartSystem = New TPartSystem

Return ps
EndFunction

Function CreateParticleSystemFromString:TPartSystem(str:String)
Local ps:TPartSystem = New TPartSystem

Local posCutLeft:Int
Local posCutRight:Int
Local varStr:String 'this is the string for each individual variable

posCutLeft = str.Find("id=")
If posCutLeft <> - 1 Then
varStr = Right(str, str.Length - posCutLeft - 3)
posCutRight = varStr.Find(";")
varStr = Left(varStr, posCutRight)
ps.id = Int(varStr)
EndIf

posCutLeft = str.Find("useGravFall=")
If posCutLeft <> - 1 Then
varStr = Right(str, str.Length - posCutLeft - 12)
posCutRight = varStr.Find(";")
varStr = Left(varStr, posCutRight)
ps.useGravFall = Int(varStr)
EndIf

posCutLeft = str.Find("useDrag=")
If posCutLeft <> - 1 Then
varStr = Right(str, str.Length - posCutLeft - 8)
posCutRight = varStr.Find(";")
varStr = Left(varStr, posCutRight)
ps.useDrag = Int(varStr)
EndIf



posCutLeft = str.Find("gravityMultiplier=")
If posCutLeft <> - 1 Then
varStr = Right(str, str.Length - posCutLeft - 18)
posCutRight = varStr.Find(";")
varStr = Left(varStr, posCutRight)
ps.gravity.MultiplyByScalar(Float(varStr))
EndIf

Local wDir:Float
posCutLeft = str.Find("windDirection=")
If posCutLeft <> - 1 Then
varStr = Right(str, str.Length - posCutLeft - 14)
posCutRight = varStr.Find(";")
varStr = Left(varStr, posCutRight)
wDir = Float(varStr)
EndIf

posCutLeft = str.Find("windIntensity=")
If posCutLeft <> - 1 Then
varStr = Right(str, str.Length - posCutLeft - 14)
posCutRight = varStr.Find(";")
varStr = Left(varStr, posCutRight)
ps.windForce = Float(varStr)
EndIf

posCutLeft = str.Find("useWind=")
If posCutLeft <> - 1 Then
varStr = Right(str, str.Length - posCutLeft - 8)
posCutRight = varStr.Find(";")
varStr = Left(varStr, posCutRight)
ps.useWind = Int(varStr)
ps.wind = TVector2d.CreateVecByDirMag(wDir, ps.windForce)
EndIf

posCutLeft = str.Find("isActive=")
If posCutLeft <> - 1 Then
varStr = Right(str, str.Length - posCutLeft - 9)
posCutRight = varStr.Find(";")
varStr = Left(varStr, posCutRight)
ps.isActive = Int(varStr)
EndIf

Return ps
End Function

EndType



Offline Ashmoor

  • Jr. Member
  • **
  • Posts: 26
Re: Need Help Optimizing Particle System
« Reply #5 on: April 29, 2019, 12:13:57 PM »
How do I add BlitzMax specific code(blue box with syntax) into replies?

I'll try to optimize what I have based on your suggestions and will post an update.

Offline Yellownakji

  • Full Member
  • ***
  • Posts: 216
  • Whoop-de-doo-ba-dooo!
Re: Need Help Optimizing Particle System
« Reply #6 on: April 29, 2019, 12:17:55 PM »
How do I add BlitzMax specific code(blue box with syntax) into replies?

I'll try to optimize what I have based on your suggestions and will post an update.

You can create a thread by:

Code: [Select]
Global Thread:TThread
Function MyFunction:object(d:object)

Thread = CreateThread(MyFunction,null)
WaitThread(Thread)

Use threads; Splitting up loads = faster rendering.

Also, i don't think you need more than 20 particles per instance.  There are lots of techniques to reduce particles and get more out of them.

Offline Derron

  • Hero Member
  • *****
  • Posts: 2087
Re: Need Help Optimizing Particle System
« Reply #7 on: April 29, 2019, 02:23:41 PM »
Only had some minutes to scan through:
{code=BlitzMax} ... so instead of {code}...{/code} you define the language - BlitzMax is one of the supported here on Syntaxbomb


Code: BlitzMax
  1.                 For Local a:TAttractor = EachIn entityList
  2.                         a.Updateself()
  3.                         If a.isDead Then entityList.Remove(a)
  4.                 Next
  5.  
You might run into "concurrent modification" - with NG things got better but make sure that the "iterator" does not get garbaged/puzzled. I described that in IWadAdam's Strata Nova thread already:
https://www.syntaxbomb.com/index.php/topic,5446.msg25595.html#msg25595

Pay attention to not fall into this trap.


Code: BlitzMax
  1.         Field useDrag:Int = True
  2.         Field useFriction:Int = True
  3.         Field useWind:Int = True
  4.         Field useAttractor:Int = True
  5.         Field useGravFall:Int = True
  6.  

If Garbage Collection creates hickups (so "average fps decreases" - with some spikes) then you might consider saving memory by using a simple "bitmask flag". All above recieve a "2^n" number ("USE_DRAG=1", "USE_FRICTION=2", "USE_WIND=4", 8, 16 ...).
Then only have a "usesFlag:int" and do your "usesFlag & USE_FRICTION <> 0" thing. "usesFlag" stores any combination of the (non-exclusive) usecases: "usesFlag = USE_DRAG + USE_FRICTION + ...").

This saves on storing some integer values for each of the particles. Think it is not needed for now.


Code: BlitzMax
  1.         timeToLive:-(1000 / (1 / frlManager.dt))
  2. 'change to
  3.         timeToLive :- (1000 * frlManager.dt)
  4.  
  5.  
  6. 'same here
  7.                 'countdown to spawn
  8.                 cometTrailSTL:-(1000 / (1 / frlManager.dt))
  9.  
  10.  


Code: BlitzMax
  1.         If isComet Then UpdateComet()
  2.  
Why not have a custom "TCometParticle extends TParticle"? saves a simple "if ..." check on "TParticle". Also this allows to have less "knowledge" coupled to "basic particles" (eg a particle does not need to know about "space", a "comet" might need to know about space, planets, ...


Code: BlitzMax
  1.         Method CheckTargetHit:Int()
  2.                 Local th:Int = False
  3.                
  4.                 If (target_xPos - curPos.dx) ^ 2 + (target_yPos - curPos.dy) ^ 2 < targetRange ^ 2 Then th = True
  5.  
  6.                 Return th
  7.         End Method
  8.  
  9. 'shorter - but might be optimized by GCC already:
  10.  
  11.         Method CheckTargetHit:Int()
  12.                 Return (target_xPos - curPos.dx) ^ 2 + (target_yPos - curPos.dy) ^ 2 < targetRange ^ 2
  13.         End Method
  14.  


I see you use "DrawImage()" - means each different particle image leads to a texture switch. Assume you have 3 Type of particles: PartA, PartB and PartC - each one uses a different particle image. Now you render 1000 particles
PartA_1
PartB_1 'switch
PartB_2 'no switch
PartB_3 'no switch
PartC_1 'switch
PartB_4 'switch
...

Texture switches are neglectible if done a dozen times per render - but with 800 particles you might have a multitude of them.
How to avoid?
1) Use a engine which auto-batches for you (partially done in gl2sdlmax I think)
2) Order by images "within layer"
3) use a texture atlas and put particles on it (all particles are on a bigger texture and each particle draws the portion of the texture belonging to its "image")


Code: BlitzMax
  1.         Method ApplyForce(force:TVector2d)
  2.                 Local f:TVector2d = TVector2d.Divide(force, mass)
  3.                 acceleration.AddVector(f)
  4.         EndMethod
  5. 'avoid vector allocation
  6.         Method ApplyForce(force:TVector2d)
  7.                 acceleration.AddVector( TVector2d.Divide(force, mass) )
  8.         EndMethod
  9.  
(might actually make no difference - and I would do similar to your approach, think this does not cost too much to justify this change)


Within your "move()" you have the "qbTrajectory" - where is that type defined -  I somehow assume that a bezier movement is rather costly (just measure a loop doing 1000 calculations of it). So precalculate on init and "cache" information if possible (does not work easily with moving targets).


Code: BlitzMax
  1.         Method Scale()
  2.                 If Not isScaling Then Return
  3.                 lastScale = curScale
  4.                 curScale:+scaleSpeed
  5.  

Your "Scale()" is not frame-rate-independend (does not utilize "dt").
Same for "Fade()", "Animate()" ... and the likes.


Generic optimization: use the object's methods instead of wrappers:
Code: BlitzMax
  1. Len(image.frames)
  2. '
  3. image.frames.length
  4.  

As said stuff like above is only a minor thing.



My assumption is this one:
Code: BlitzMax
  1.         Field qbTrajectory:TQuadraticBezier             'make sure path is parametrized by arc len
  2.  

Maybe have "trajectory" be a basic "path"-object - doing linear interpolation between posA and posB.  and if you need the bezier, then assign a more complex "path"-object.
So simple rain does not need the complex calculations.


Edit: might be of interest for you:
http://ammarhattab.com/resources/papers/BezierCurves.pdf

TCatmullRomSpline:
https://github.com/TVTower/TVTower/blob/master/source/basefunctions.bmx
(extended some existing code - shows how to "cache").

bye
Ron

Offline Ashmoor

  • Jr. Member
  • **
  • Posts: 26
Re: Need Help Optimizing Particle System
« Reply #8 on: April 30, 2019, 12:58:39 AM »
"You might run into "concurrent modification""

I have never seen this happen. I am aware of it for arrys. Should I just flag the dead particles and remove all of them at start of next update cycle? I have no idea under what circumstances the list may fail.

"If Garbage Collection creates hickups"

I'm a bit uneasy about using bitmask flags now as I am nearing the end of this dev cycle.

"Why not have a custom "TCometParticle extends TParticle"?"

This makes sense, I didn't know how to think about this. Your "knowledge" based approach makes sense, especially since most particles are simpler than comets.


"Texture switches are neglectible if done a dozen times per render - but with 800 particles you might have a multitude of them."

This is beyond me at the moment. I have to learn about it. How would I draw different parts of an atlas? Using pixmaps directly from the atlas? This optimization makes a lot of sense, especially in scenes with 20+ different particle types.

"Within your "move()" you have the "qbTrajectory" - where is that type defined -  I somehow assume that a bezier movement is rather costly (just measure a loop doing 1000 calculations of it). So precalculate on init and "cache" information if possible (does not work easily with moving targets)."

The bezier path is parametrized by arc len into an array of points and the position just iterates through that array, does not calculate position on path at each frame, it's all cached on creation. For most particles the qbTrajectory is just null.

"Your "Scale()" is not frame-rate-independend (does not utilize "dt").
Same for "Fade()", "Animate()" ... and the likes."

Yes, I didn't implement it yet.

I disabled the following code and got to ~3.6k particles without dropping frames.

Code: BlitzMax
  1.  
  2. If Self.useGravFall And p.useGravFall Then ApplyGravFall(p)
  3. If Self.useDrag And p.useDrag Then ApplyDrag(p)
  4. If Self.useWind And p.useWind Then ApplyWind(p)
  5. If p.attractorGroupId <> - 1 Then ApplyGroupAttractor(p)
  6. If p.attractorId <> - 1 Then ApplySingleAttractor(p)
  7.  

 Do you have any idea how many would be an indication that the code is running well?

Offline Derron

  • Hero Member
  • *****
  • Posts: 2087
Re: Need Help Optimizing Particle System
« Reply #9 on: April 30, 2019, 05:36:30 AM »
Just do no physics/movement at all. Just lifetime, deletion and render. Also render all of them with the same image might show a potentially reachable limit.
This the barebone limit.


Texture atlas: drawsubimagerect() is the command you have to look for.


Bitmask: would mean to just replace the occurrences of variables in your code .. not needed for now as there is more important stuff


Disabled stuff: reenable step by step to see what cuts how many.


Bye
Ron

Offline Henri

  • Full Member
  • ***
  • Posts: 198
Re: Need Help Optimizing Particle System
« Reply #10 on: April 30, 2019, 07:31:11 AM »
Hi,

you can time your code parts with Millisecs().

Code: BlitzMax
  1. Local start:Int = MilliSecs()
  2.  
  3. ' Executing some code..
  4.  
  5. Print "MyFunction took " + (MilliSecs()-start) + " ms."

-Henri
- Got 01100011 problems, but the bit ain't 00000001

Offline Derron

  • Hero Member
  • *****
  • Posts: 2087
Re: Need Help Optimizing Particle System
« Reply #11 on: April 30, 2019, 09:13:00 AM »
If you use Henri's suggestion:
- do not include the "DrawXYZ"-commands - some engines are intermediary ("DrawCommand does already transfer to GPU") and some not (in that case "flip" needs some milliseconds)
- pay attention to millisecs being ... milliseconds, so if your algorithm needs to run 1000 times to take 1ms you will have to do something more to measure it.

For algorithm to (simple) test just have a for loop doing it eg. 10000 times. Do this loop 2-3 times to average out stuff. Maybe even have GCCollect() to enforce garbage collection after each "2-3 times" loop.


bye
Ron

Offline Ashmoor

  • Jr. Member
  • **
  • Posts: 26
Re: Need Help Optimizing Particle System
« Reply #12 on: May 01, 2019, 02:08:43 AM »
Using just the barebone spawn - count down and die I get around 5.5k particles before dropping the frame rate. The cpu load never goes higher than 20%, 5% being my 20 chrome tabs. Got 32 GB ram but my game uses max around 300mb. I am guessing that I only engage 1 core.

Looks like this will keep me busy a few days, timing my code, learning about threads etc.

Offline Derron

  • Hero Member
  • *****
  • Posts: 2087
Re: Need Help Optimizing Particle System
« Reply #13 on: May 01, 2019, 06:26:00 AM »
Did you use only a single image or multiple images for rendering? Using only one can be used to simulate single-textures with no further texture switches.

20% single core usage means threading wont help on your computer ..you are not cpu limited.


5 5k particles means 5500 draw calls


Bye
Ron

Offline GaborD

  • Sr. Member
  • ****
  • Posts: 259
Re: Need Help Optimizing Particle System
« Reply #14 on: May 01, 2019, 07:42:42 AM »
Yeah agree to Derron, sounds like the main bottleneck is probably constantly pushing all the data from the CPU to the GPU.

Do yo need to stick to stock BMX due to project constraints or are you free to fully leverage 3D hardware?
If you think about it, particles are as parallelized as it gets, you call simple functions tons of times. They basically beg for a GPU implementation.
Could easily run 1 million particles at hundreds of FPS on midrange hardware that way.
Obviously you won't need that many particles for a game, so you can speed it up a ton more by going with 64K or such and thus have basically no performance impact from the positioning/movement calcs (if you keep them reasonably simple) and also get much faster rendering due to not streaming tons of unique geometry or lots of separate tiny objects to the GPU.

I admit I never tried it in BMX, but I have tested around quite a bit in AGK. I am sure it could be done in similar fashion in openB3D or whatever other BMX mods supply the needed functionality (all you need is simple shaders and rendertextures, nothing fancy)
I know this is an out there approach not everyone will like, just pointing to a possible additional solution.