Christmas Project - Streaming Terrain Editor

Started by Kronos, December 18, 2019, 22:24:57

Previous topic - Next topic

Kronos

As a Christmas project I have decided to attempt a streaming terrain editor/object placer using OpenB3D and C#.  I wrote some code in Blitz3DSDK a long time ago which could load meshes in bit by bit which seemed to work ok so I am using this for the time being. I kinda wanted to do this in Python as an excuse to learn the language but have had a few issues getting OpenB3D to work with it so I will stick with C#.

Not sure how I am going to do the terrain painting yet but as this is a quick and dirty project I may be able to get away with using vertex colours.

I am using OpenTK to create the opengl context. Took a bit of time to get it working in a form but its all good now.
Having some problems getting the lighting to work properly in OpenB3D, dunno whether the normals in the meshes are messed up or whether I have forgotten some setting I need for lighting. Also could be my OpenB3D wrapper. It's been a long time since I dabbled with Blitz3D/OpenB3D... EDIT: definitely not the normals, I visualized them and they are all pointing in the expected directions.

Current progess: I can select vertices in any terrain mesh and raise them --woohoo!


3DzForMe

BLitz3D, IDEal, AGK Studio, BMax, Java Code, Cerberus
Recent Hardware: Dell Laptop
Oldest Hardware: Commodore Amiga 1200 with 1084S Monitor & Blitz Basic 2.1

Kronos

#3
Progress report.

Well this has certainly caused me some headaches.. I have abandoned the streaming side of things for the time being and am concentrating on getting the editor to work. I have changed how I save the terrain information from text files to images.
Annoyingly UpdateNormals in Openb3d doesn't seem to work on meshes where I have amended vertex positions until I save them and then reload them which makes terrain sculpting a bit tricky unless you are in wireframe mode.

The saved meshes appear to be losing resolution at some point. Will need to look into this. Floats converting to ints somewhere no doubt.
Also some bug with the wireframe checkbox which keeps switching on.
Will need to manually calculate the normals on the edges of each tile so no seams will be visible.

Also one of my monitors just died. Forgotten how tedious it is having to work on a single screen.

Alot happier now anyway than I was a couple of hours ago:) Here's a vid of current version.








Qube

Looks like it progressing very well, nice one :)
Mac Studio M1 Max ( 10 core CPU - 24 core GPU ), 32GB LPDDR5, 512GB SSD,
Beelink SER7 Mini Gaming PC, Ryzen 7 7840HS 8-Core 16-Thread 5.1GHz Processor, 32G DDR5 RAM 1T PCIe 4.0 SSD
MSI MEG 342C 34" QD-OLED Monitor

Until the next time.

3DzForMe

QuoteA lot happier now anyway than I was a couple of hours ago:) Here's a vid of current version.

Brilliant looking work, you're inspiring me to crack open some of my old coding exploits  ;D
BLitz3D, IDEal, AGK Studio, BMax, Java Code, Cerberus
Recent Hardware: Dell Laptop
Oldest Hardware: Commodore Amiga 1200 with 1084S Monitor & Blitz Basic 2.1

Kronos

#6
Just time for a quick update today. Changed how the heights are saved in the image so that saved tiles no longer look chunky. I convert the mesh float height value to 3 byte values in the heightmap image(rgb) which retains decimals to 3 places. Hopefully should be enough definition.
Next on the list is to look at calculating tile edge normals. Edit..not looking forward to doing this so maybe I'll do something more fun first

Kronos

#7
Might need to rename this to "Long term" project as it is obviously going to take some time. Don't want to spend 100% of my free time working on this I just want to be able to do a couple of hours every now and again.

Had a devil of a time getting shader/texturing to work. As per normal it turns out the problem was in an entirely different place to where I was looking. Was thinking I had screwed up the shader somehow but it turns out when mesh is loaded the UV coords were being calculated as ints so all had value of 1 or 0! As an aside, looking at recent shader examples it looks like shader code format has changed yet again since I last looked at shaders. I am a bit of a shader novice. Anyway as you can see in vid below it is sort of working now.

In order for picking and lighting to work properly on any altered mesh it has to be deleted and reloaded, so next job will be to create a "refresh" function that will re-generate all currently visible tiles. I haven't "delta-timed" anything yet so should probably do that sooner rather than later. Seems to be alot more stable now which is good.

Finding that I am having to create alot of files to keep track of all the elements. I have a main header file, a log file, then for each tile I have a tile header file(currently just contains info about textures to use for this tile), heightmap,  colour map, normal map(not done yet). Not sure how I am going to store vegetation and entity info at this stage.

Anyway lots to think about  :)



Kronos

#8
Still putting off doing the edge normal calculations as I think it will be horrible.
Spent a couple of hours today just cleaning up the current code a bit.
Added a refresh button so that normals can be re-calculated on altered meshes on the fly. Ideally this would happen after every mesh update but I am a bit wary of constantly creating and deleting entities. However it is a bit annoying that you can't easily see the deformations as you do them due to the lighting.
Re-jigged the texture loading slightly as I discovered that I was constantly reloading same textures. This has smoothed things out a bit.
The texture painting for multiple tiles is more or less sorted.
Still getting an odd effect with the alpha channel when painting textures. Instead of the texture fading in gradually the area goes bright and then the texture fades in.
Looking forward to being able to paint vegetation.
However thats enough for today, time to go play something.

Kronos

#9
Porting existing code over to Agk. This is contingent on me being able to get mesh creation via memblock to work which currently it isn't :( No crashes just nothing appearing on screen.

EDIT: that was a major pain in the butt to get working but I appear to have done it! I can now create meshes in AGK. >:(


Kronos

#10
This weeks progress..

Whilst porting code over to AGK I made the decision to scale back my ambitions somewhat so this is no longer going to be a streaming Terrain Editor as it was not working well and was causing me too many headaches. So it will just be a plain single terrain editor. This gave me an excuse to re-write alot of my previous code and I am alot happier the way its all laid out now. Vertex painting is working but as I haven't added the splatting shader yet all you see is textures. I now store all terrain tile info in text files instead of heightmaps which means I can dig deep holes in the terrain as well as build high mountains.

Next on to do list is add the splatting shader, fix those bloody edge normals(should be easier now!), add some water, and then add vegetation and objects.
As I navigate round the map using WASD and space bar keys one issue I am having is key inputs being picked up by the form controls, not sure how to fix this at the moment. Any suggestions would be welcome. Specifically after amending one of the sun/skybox controls and then pressing a key as control is  selected it doesn't like it. You can see in the video every time I change a control setting I click back to the terrain edit panel so as to de-select the control.




3DzForMe

#11
Oh my, impressive progress ;)  Have you any thoughts regarding using a heightmap as the base load for your editor, then tweaking the terrain from there - I sincerely mean it when I say exciting stuff.  :D

If you were to use a 'heightmap', would you be inclined to make the 'ground' of the terrain at 128 pixels - giving the option for digging down 128 graduations & up by the same amount? Or..... set the 'ground' wherever you want between 10 to 245 graduations.... the mind boggles. Sorry for my rambling.
BLitz3D, IDEal, AGK Studio, BMax, Java Code, Cerberus
Recent Hardware: Dell Laptop
Oldest Hardware: Commodore Amiga 1200 with 1084S Monitor & Blitz Basic 2.1

Kronos

#12
When I was using heightmaps before I was taking the height as 4 bytes,(pixel colours) and converting to a float to get a bigger range and +- values.

I may return to heightmaps as I am having some speed issues loading and saving to text files but have other problems to solve first.

Edit:ooh i think I just fixed my shader problem.

Kronos

#13
While we are rambling, it's amazing how one little success can totally re-energise you. After yesterdays round of coding I was pretty despondent, many things were/are not working, I seemed to have coded some massive memory leak somewhere, couldnt get the terrain shader to work, program runs real slow on large maps etc. I was in one of those "am I wasting my time doing this, should I give up" moods.

However today after only 10 minutes I got the shader working and it's like a little ray of sunshine which has totally transformed my outlook on the project again. It's good to have a win now and again. It also makes me happy that I can pull together all those little experiments I have been coding for years and make use of them in one project. No code is ever wasted in the long run.  :)

ramble ends.

Kronos

I have identified the source of the memory leak issue when saving files and have a workaround that fixes the issues. The GetMeshMemBlockVertexRed/Green/Blue/Alpha commands seem to be the culprits.Whether this is a wrapper problem or an AGK problem I don't know.

Also found a way to dramatically increase picking speed of terrain which means the selection area can now be much bigger. Before I was pushing memblock data to the mesh after every update while the left mouse button was pressed if a change was detected. Changing this to only pushing memblock data when left mouse button is raised after a press means there is no slowdown. For terrain sculpting this is not an issue as I have the vertex selection lines to use as a guide which work off the memblock, however terrain painting has become a bit more vague as you don't see anything until mouse button is raised. The pro's outweigh the con's so I can live with it. Makes terrain painting feel more like real painting anyway.

My first attempt at fixing the tile edge normals doesn't seem to have worked so will need to re-examine when I am feeling fresh.