Jul -- Game Development/Mod Projects: Datamijn

Sanqui
2060

💛🤍💜🖤🦉
Level: 87

Posts: 1717/2066
EXP: 6305062
For next: 87712

Since: 12-20-09

Pronouns: any ✨
From: Czechia | Estonia

Since last post: 6 days
Last activity: 1 day

Posted on 05-04-18 08:07:22 AM

Link | Quote

*Sanqui flops, wings involved.

level 76

exp 3933688

exppct 59.6%

numposts 1717

I'm working on an awesome binary data description language and YOU get to be a part of it!

Datamijn is primarily a domain-specific language for describing binaries. It's meant to be as concise as possible. You know what the data looks like and you just want YAML out of it. Datamijn won't make you write any boilerplate.

A datamijn definition file is natural to write and read. Example:



                            position {

                            x u8

                            y u8

                            }

                            

                            _start {

                            version u16

                            positions [8]position

                            }

In this .dm file, we describe a type, "coords", consisting of two bytes. We then describe how to parse the binary from the beginning.

Is position only used once? No need to pollute your namespace. In datamijn, you can always define a type anonymously.



                            version u16

                            positions [8]{

                            x u8

                            y u8

                            }

In this example, we haven't defined _start, so datamijn starts parsing the file from the first field.

A binary file parsed with such a definition might end up looking like this pleasant YAML.

What can datamijn actually do?

Here's a more complex example in which I'm parsing a bunch of data from Final Fantasy 1... including strings!

ff1.dm:

ff1.yaml:

string_ptr is of particular note: it involves reading a NES pointer, doing some calculations to convert it into an absolute pointer within the NES ROM, and then reading a zero-terminated string at that location. The idea is that you won't even have to write this pointer math: nesptr should be a part of the standard library!

The end goal is that you'll describe a ROM according to what you know... and get all of the data out of it, without having write boilerplate or possibly even touching a programming language. Of course, this is all a lie, because if you'll be decompressing some data using datamijn, you'll probably have to write a for loop or two. But the idea is it should be all super concise and intuitive.

Support for writing is also on the roadmap, but only as a second class citizen. It's likely there will be some limitations.

Datamijn isn't complete and I won't be releasing it until I deem it usable, but it's already quite powerful and evolves with my own needs.

Is datamijn interesting to you? Do you have any questions or ideas? I'll probably follow this up with some thoughts on how to do decompression, but this should be enough for now.

____________________

In our brief lives, • we've managed to meet. • Treasure this gift, • this precious time that we have.

Register - Login
Views: 99392553	Main - Memberlist - Active users - Calendar - Wiki - IRC Chat - Online users Ranks - Rules/FAQ - Stats - Latest Posts - Color Chart - Smilies	04-24-22 10:10:43 AM

Query execution time:	0.076778 seconds
Script execution time:	0.009335 seconds
Total render time:	0.086113 seconds