SyntaxBomb - Indie Coders

General Category => Worklogs => Topic started by: TomToad on May 12, 2018, 11:22:51

Title: ForWord - Code a game competition - May 4th to June 30th
Post by: TomToad on May 12, 2018, 11:22:51
Puzzle/endless catagory.

You get letters.  You must form words within the time limit. Used letters are replaced by new letters. Letters have point values. Bonus points based on length of word, letter position, and using all letters in a word. Game will be more interesting than my description.
Title: Re: ForWord - Code a game competition - May 4th to June 30th
Post by: round157 on May 12, 2018, 13:18:08
Quote from: TomToad on May 12, 2018, 11:22:51
Puzzle/endless catagory.

You get letters.  You must form words within the time limit. Used letters are replaced by new letters. Letters have point values. Bonus points based on length of word, letter position, and using all letters in a word. Game will be more interesting than my description.


Hi, this gameplay sounds quite entertaining. Will you show us some screenshots soon? What language is used to write this game? Thanks.

Good luck to you.
Title: Re: ForWord - Code a game competition - May 4th to June 30th
Post by: TomToad on May 16, 2018, 11:59:38
Discovered some interesting stuff while creating this game.  Downloaded the Enable word list form http://www.puzzlers.org/dokuwiki/doku.php?id=solving%3awordlists%3aabout%3astart#scrabble_tm_dictionaries (http://www.puzzlers.org/dokuwiki/doku.php?id=solving%3awordlists%3aabout%3astart#scrabble_tm_dictionaries).
Then I stripped all the words less than three letters and more than seven.  Then I created a frequency list and used that to determine the point value for each letter based on how often it appeared in the list.  The below is sorted, least frequent first.  Was surprised to discover that the letter j appeared less often than the letter x or z.
q:0.0017658473685380626:10
j:0.0032268002274004884:10
x:0.0035920384421160951:10
z:0.0048052645118670662:10
v:0.0095343053963152236:9
w:0.013056472179963984:9
k:0.015702067248295291:8
f:0.015959321990834108:8
y:0.020434284117219235:8
b:0.023549924887967148:8
h:0.024956886010741180:8
m:0.029028498108701246:7
g:0.029679574926237760:7
p:0.030273484023210093:7
c:0.034500719360483767:7
u:0.037997478268326224:6
d:0.042278705341688289:6
n:0.054833371974477789:5
t:0.054855603865808303:5
l:0.055081098763589242:4
o:0.060658127503072763:4
i:0.070230544713097448:3
r:0.071358019202002146:3
a:0.082105550668068336:2
s:0.093094456954294413:1
e:0.11744155394568431:1


So I thought I'd try it on the original Enable file.  J beat even Q for least frequent letter!
j:0.0015898990156251989:10
q:0.0016140945152622664:10
x:0.0029352961401810842:10
z:0.0047435913762145501:10
w:0.0074426630330968963:9
k:0.0084767022807442031:9
v:0.0097819858137965284:9
f:0.012311052249544742:9
y:0.016323684847249990:8
b:0.018360563882486278:8
h:0.023235957059355380:8
g:0.027012365173761890:7
m:0.028353941956269817:7
p:0.029379067072471888:7
u:0.032663287786366471:7
d:0.033832949176716288:7
c:0.040864288715983035:6
l:0.053119946005832391:5
o:0.065923822379563721:3
t:0.066834973958001712:3
n:0.068461166223082501:3
r:0.070716441478727063:3
a:0.075671425114928623:2
i:0.090073477912055722:1
s:0.095160900072586499:1
e:0.11511645676009526:1

So I tried it on the Webster New International Dictionary word list. http://www.puzzlers.org/dokuwiki/doku.php?id=solving%3awordlists%3aabout%3astart#ni2--webster_s_new_international_dictionary_second_edition (http://www.puzzlers.org/dokuwiki/doku.php?id=solving%3awordlists%3aabout%3astart#ni2--webster_s_new_international_dictionary_second_edition).  J is still at the top of the list.
j:0.0011949968149878279:10
q:0.0016394171180824746:10
x:0.0030678468195442584:10
z:0.0036850972405090454:10
w:0.0060625214073668724:9
k:0.0069684205706446473:9
v:0.0088753876893780410:9
f:0.010612218146623685:9
b:0.017507466485546689:8
g:0.020657912634150961:8
y:0.023068556096391621:8
h:0.028312266763511389:7
d:0.030126309637052262:7
m:0.030831995209238915:7
p:0.034041697398255806:7
u:0.039057811001063467:6
c:0.045275655059814936:5
l:0.057976648630983459:4
s:0.061483977750254645:4
t:0.067808213336110590:3
n:0.070762037168800251:3
r:0.071758167120888158:3
o:0.076212695067967129:2
a:0.088170743586992575:1
i:0.089840237634678424:1
e:0.10500170361116186:1


Thought this was very interesting. :)

@round157:  I am planning on using AGK2 to program the game as I believe this would make a great game for mobile platform and I have found AGK2 to be the easiest for creating multi-platform games.
Title: Re: ForWord - Code a game competition - May 4th to June 30th
Post by: Rooster on May 16, 2018, 20:59:37
Wow, that's interesting. :o
Title: Re: ForWord - Code a game competition - May 4th to June 30th
Post by: round157 on May 17, 2018, 01:48:32
So according to the Webster New International Dictionary word list, the top 5 are e, i, a, o, r?

Four of them are vowels!
Title: Re: ForWord - Code a game competition - May 4th to June 30th
Post by: TomToad on May 20, 2018, 21:35:55
Currently working on the interface.  The image is what I have so far.  I am aiming this game for mobile as I think it will work best for that, which is why I have the resolution like I do.
(https://www.syntaxbomb.com/index.php?action=dlattach;topic=4529.0;attach=973;image)
Title: Re: ForWord - Code a game competition - May 4th to June 30th
Post by: round157 on May 20, 2018, 21:50:16
Hi, I like the interface of this game.

This user interface is tidy and not too complicated.
Title: Re: ForWord - Code a game competition - May 4th to June 30th
Post by: Xerra on May 20, 2018, 22:58:42
Are you planning to put this game just on mobile or have a desktop build as well? Just thinking for when it's judging time and some people might not be able to try it?
Title: Re: ForWord - Code a game competition - May 4th to June 30th
Post by: TomToad on May 21, 2018, 00:01:45
There will be a windows version. Rules of the contest require it :)
Title: Re: ForWord - Code a game competition - May 4th to June 30th
Post by: Qube on May 21, 2018, 09:23:27
Quote from: TomToad on May 21, 2018, 00:01:45
There will be a windows version. Rules of the contest require it :)
Windows version or an online version as minimum :)

Word games are what I play for 10 minute quick blasts so I'm quite curious to see how this turns out. Don't forget to add the words SyntaxBomb and Qube in :P
Title: Re: ForWord - Code a game competition - May 4th to June 30th
Post by: Derron on May 21, 2018, 11:33:11
Hope you add enough polishing-time then. Such "simple to play, hard to master"-games really benefit from polishing. Do not make it look like Minesweeper in Windows 9x.

- if you have time, add "swirling/curling" letters (so they are not 100% visible all the time)
- add some particles when scoring/removing letters
- add 3rd dimension so you can properly have a "qube"-board (joke)
- if you like, make it cute (for children to play) - add some eyes to the letters (overlay, so easy to animate)
- ...
such games have huge potential as they are perfect "time killers" - but as there are plenty of them, you need to think of a special "twist" to make it an interesting play, and you need to invest (time) in visuals to make it look nice.


bye
Ron
Title: Re: ForWord - Code a game competition - May 4th to June 30th
Post by: TomToad on May 21, 2018, 13:41:33
Definitely will be adding a lot of polish over its development. Don't know how much I'll get done by the deadline.  I will be adding more elements to the main interface, such as score, bonus points, menu, etc..  right now,I have finished with keyboard input and checking for valid words.  Now I need to create touch input for mobile and replace used letters with new ones.

Once I get the main part of the game working, then I'll add the polish.
Title: Re: ForWord - Code a game competition - May 4th to June 30th
Post by: TomToad on May 24, 2018, 13:56:39
Hit a bit of a snag, came up with a solution. 

The game was coming along great on my PC, but when I tried it on my android, a part of it was too slow.  The part giving me problems was a function that would replace the used letters with new ones.  I didn't want to just pick letters at random as I wanted to make sure that the letters will be able to make at least 1 new word.  So I created a list with 7 letter words.  The function would check each word in the list and see if it contained all unused letters.  If it did, it was added to a new list, then one of the words on the new list would be chosen randomly.

My PC is fast enough for this method.  It would only take 2-3 seconds to choose a new word, which I could hide easily enough with a short animation before the next set of letters show up.  On my Android, the delay could be 6-7 seconds, which lasts well beyond the animation (I don't want the animation too long as that would just be too annoying).  This leaves a very noticeable pause in the game.

After reading about inverted index https://en.wikipedia.org/wiki/Inverted_index (https://en.wikipedia.org/wiki/Inverted_index), I thought of a new strategy for determining new letters. I would create a second database containing 26 entries, one for each letter.  Each entry would consist of a list.  The list will contain the index of all words that contain that letter, and the number of times the letter appears in the word.  Now to check the remaining letters against this database and copy the indices that appear in all sets.  For example, if I want to find all the 7 letter words that contain "bstt", first I make a set containing all the entries from the b set.  Then I check this set against the s set, removing any entries not found.  Then I check this one against the t set, if the entry is not found, or the number is less that 2, then it is removed from the set.  The resulting set is all entries containing at least 1 b, 1 s, and 2 ts.

Now I just need to figure out exactly how I am going to implement this idea.  Hopefully it will prove to be fast enough as I won't need to check every word, I can better optimize the searches, and I no longer need to use Mid() to check letters individually as I only need to check against indices.

Here is the game so far on my PC before adding the new optimizations.  I will be adding more features as well.  For example, there will be a time limit forcing you to find words quickly.  Bonuses will appear randomly giving letters extra points, multiply word values, and increase time limits.  The game will progressively get more difficult as you score higher,  requiring you to find longer words and giving you less time.
Title: Re: ForWord - Code a game competition - May 4th to June 30th
Post by: Derron on May 24, 2018, 16:46:21
(https://i.imgur.com/7sSzKo0l.png)
video not playable.


@ Mid()
You know you can access a string's letters in BlitzMax using "mystring[index]" ? It returns the numeric keycode for the letter (so the equivalent of Asc("h") - or the opposite of Chr(13) which is "enter").

That way you could do your lookups pretty easy.


There is another speedup thing: caching.
on each letter you add, you store what you have "for now" so you do not need to lookup these things each time. Adding a letter can reuse the cache, removing a letter invalidates the cache.
Same can be said for the lookup: if you know that the "up to now used letters" are not forming a word beginning with one of these letters, you might not have to lookup these ones again. This step is a bit more advanced and should really only be taken if the simple cache + lookup-speedup do not improve your "word lookup"-times.



bye
Ron
Title: Re: ForWord - Code a game competition - May 4th to June 30th
Post by: Matty on May 24, 2018, 20:02:25
There is an alternative method that is not perfect but should work well enough:

You have 7 letters.  'N' remain.  'M' need to be replaced. M = 7 - N.

Each letter of the alphabet has a score indicating 'frequency of occurring in a 7 letter word' (precalculated).
Excluding the N letters that already exist in the puzzle add letters 1 at a time with no repeats using the highest frequency letters.

Then you only need to check 26 'scores' M times.

It won't be perfect but it will most of the time give you letters to create a seven letter word.

You can also put a second score in of most often used letters to reolace that is generated at runtime to prevent the same words coming up over and over.
Title: Re: ForWord - Code a game competition - May 4th to June 30th
Post by: TomToad on May 24, 2018, 22:59:05
@Derron, I'm not using BlitzMax, but rather AGK2 which doesn't allow you to index strings so easily. I have thought of a couple more optimizations though. One would be to load the words into a memblock, that would allow me direct access to the strings.
Also,I have ordered the letters inn each word alphabetically, so once one letter matches the next input needs to start where the last one left off. An idea would be to take advantage of this order by creating another table containing how many letter each word shares with the last, that way, when a word fails to match,I only need to start from the last matching letter that falls within the shared letters, instead of starting the checks from the first letter.

This might hopefully be enough. My other idea had a few problems. One being that the inverted index will take up about 4x as much space as the original index. Second, implementing a function to remove elements from a set would prove to be very difficult. Removing from an array would be slow, probably negating any speed advantages I would gain. Linked lists are non existent in AGK2, and without pointers, implementing my own linked list would be very difficult.
Title: Re: ForWord - Code a game competition - May 4th to June 30th
Post by: Qube on May 25, 2018, 00:25:10
Are you sure that AGK doesn't have the needed commands? https://www.appgamekit.com/documentation/Reference/Core.htm

Scroll down to the Strings section and you'll see it's quite feature complete.

In a nutshell, is what you are trying to achieve with speed :

You have 7 letters and any word that can be made from the word list ( max 7 letters ) is put into a new list.

If that's correct then I'll happily whip up a speedy routine for you. No way should it be 2-3 seconds on a desktop to do that :o

Title: Re: ForWord - Code a game competition - May 4th to June 30th
Post by: Derron on May 25, 2018, 07:33:08
Quote from: Qube on May 25, 2018, 00:25:10No way should it be 2-3 seconds on a desktop to do that :o

This depends on the size of the word list :-)

But I am with you: 2-3 seconds sounds as if something is done _very_ wrong.


Maybe TomToad could describe what he wants - in headwords. Also provide the language file for the 7 words-thing. While not all languages offer the same abilities, the basic logic to tackle something might be translateable into "AGK2" then.


@ not-repeating words
Just cache what you have used for now. Limit that history to X entries (loop: "remove first entry" until history count is below limit).
If you cannot come up with a new word ("empty") you remove an entry from history - even if this means you are repeating a word more often than desired.


bye
Ron
Title: Re: ForWord - Code a game competition - May 4th to June 30th
Post by: TomToad on May 25, 2018, 11:59:16
Fixed the youtube video above.

I do think that using Mid() is causing a huge slowdown.  Plus, I am only doing searches for half a frame, using the other half to update particles, move sprites, render screen, etc...

For those who might be confused, let me try and explain what I'm doing a little better.  The game presents you a ring with 7 letters, You need to find a word, 3 letters or more, that will score the highest with the given board.  Each letter has a point value.  Each position on the board has a multiplier (seen across the top), so you can score higher by having higher point letters later in the word.  The board will also contain random bonuses that will affect the final score.  After the player creates a word, the letters used will be removed from the ring and replaced with new letters, similar to how letters are replaced in Scrabble.  Now the trick comes, I don't want to replace letters with just random ones.  I want to, as much as possible, replace letters so that a 7 letter word is always possible.  This becomes more important on higher levels where you must find longer words.  So I need to check with the list of 7 letter words and pick one randomly that contains all remaining letters.  So if the player has aberstt and plays "star", then I need to find a word that contains bet, such as "tablets".

I might go back to the inverse index idea.  After finding a better algorithm for set intersection, I have greatly decreased the number of worse case checks from about 40,000,000 to about 36,000 (yes, that is about 99.1% drop in checks).  So the number of worse case from 1 letter to 4 letters is 0 to 36,000.  Whereas Worst case for the original method is 135,000 to 136,000.  Best case for old method, 9000-19500, best case for new method is 0-2,500.  As for the problem I had with AGK2 not having linked lists, I found a suitable way around it using arrays. Plus, as I will be comparing indices instead of strings, I can completely get rid of the slow Mid() command.

Right now, I'm expecting a busy weekend, so I might not get back to this until Monday.



Title: Re: ForWord - Code a game competition - May 4th to June 30th
Post by: Derron on May 25, 2018, 17:51:49
Short reminder:
Do not just check if the new letters can build a 7-letter-word. Also make sure you can at least build a 3 or 4 letter word (depending on what minimum the "level" sets). Else you will play a level requiring a 3-letter-word but you only add letters allowing to create a more difficult 7-letter-word.


bye
Ron
Title: Re: ForWord - Code a game competition - May 4th to June 30th
Post by: TomToad on May 29, 2018, 14:06:43
Implementing the inverse index turned out to be no easy task, but I did finally get it done.  Wow! What a difference it makes.  On my PC, the search is practically instantaneous.  On both my androids, there is a very slight pause, maybe 6 or 7 frames at most.

A couple of things I learned about AGK2 while I was writing the code.
1.  Arrays in AGK2 are very efficient when inserting and removing elements.  It makes me think that internally, they behave more like linked lists than just blocks of memory.

2. conditionals are not short circuited. So "If a() = 1 and b() = 1", both a() and b() will be called, even if a() returns false.

3.  You cannot insert into the head of an empty array.  So doing
        Array As Integer[]
        Array.Insert(100,0)
     is not valid.  You have to break it into two parts. 
         If Array.Length <> -1
             Array.Insert(100,0)
         Else
             Array.Insert(100) //inserting at the end of an empty array is ok
         EndIf
Title: Re: ForWord - Code a game competition - May 4th to June 30th
Post by: Derron on May 29, 2018, 20:42:16
Quote2. conditionals are not short circuited. So "If a() = 1 and b() = 1", both a() and b() will be called, even if a() returns false.

Really? That would be a little bummer.


bye
Ron
Title: Re: ForWord - Code a game competition - May 4th to June 30th
Post by: Steve Elliott on May 29, 2018, 20:44:26
These kind of efficiency savings through working the problem, give me goosebumps lol.  The days of massive advances in processor speed have halted.  We now have to go old skool and use our brains to produce greater speed.  Well done Tom!
Title: Re: ForWord - Code a game competition - May 4th to June 30th
Post by: TomToad on June 23, 2018, 11:47:12
haven't been able to work on this for the last few weeks.  Next week looks to be pretty hectic as well. :(  I might have a few days before the deadline to work on this.  One last push and hopefully get something I can submit.