Documentation#

Quick Start#

Prepare The files#

First thing we need to do is to set up a pristine copy of the mod sources. This module works in the following way:

  • Get a file from an unchanged mod (we’ll refer to it as source mod).

  • Parse, convert, apply edits.

  • Format back to ndf and write it to a directory with our final mod (we’ll refer to it as a destination mod).

Generate a new source mod using CreateNewMod.bat provided by Eugen Systems and place it to path/to/src/mod.

Make The Script#

Create a new python file, say, my_mod.py with the following code:

import ndf_parse as ndf

# change PATH_TO_SRC_MOD and PATH_TO_DEST_MOD to actual paths, like:
# ndf.Mod(r"C:\game\mods\src_mod", r"C:\game\mods\dest_mod")
# `src_mod` must be a root folder of the source mod, i.e. the one where
# folders CommonData and GameData reside.
mod = ndf.Mod(PATH_TO_SRC_MOD, PATH_TO_DEST_MOD)
mod.check_if_src_is_newer()
with mod.edit(r"GameData\Generated\Gameplay\Gfx\Ammunition.ndf") as source:
    for obj_row in source:
        obj = obj_row.value

        # skip anything that is not of this type
        if obj.type != "TAmmunitionDescriptor": 
            continue

        # NOTE EMBEDDED QUOTES IN COMPARED STRING!
        if obj.by_member('Caliber').value != "'XUTVWWNOTF'": 
            continue
        
        # print namespace of filtered object. Note the use of view again.
        print(obj_row.namespace) 

        # grab a row by member name
        cursor_row = obj.by_member("WeaponCursorType")
        
        # print the value of `WeaponCursorType = ...`
        print(cursor_row.value)

        # set the value
        cursor_row.value = cursor_row.value.replace('Cursor', 'Replaced')

        # print again to view changes
        print(cursor_row.value)

Run this script (but don’t forget to substitute your mod paths first!) You should see prints that correspond to operations from this sctipt. Now, if you navigate to your newly generated mod and check Ammunition.ndf, you should find that some items have their “Weapon_Cursor_MachineGun” replaced with “Weapon_Replaced_MachineGun”.

Step-by-step Explanation#

Now let’s go line by line and examine what this code does.

mod = ndf.Mod(PATH_TO_SRC_MOD, PATH_TO_DEST_MOD)
mod.check_if_src_is_newer()

Here we initialize our Mod object. It’s nothing but a convenience wrapper that saves you time from writing boilerplate code. Second line here checks if your source mod was updated. Whenever the game gets an update - you should delete your source mod and regenerate it anew. When next time you run the script, that second line detects it (by comparing modification date of source and destination folders), nukes destination mod and makes a new fresh copy of it from the source.

Warning

Never store anything important inside of source and destination folders! It will get nuked by such an update. Store it elsewhere or regenerate it with a script.

Edits#

with mod.edit(r"GameData\Generated\Gameplay\Gfx\Ammunition.ndf") as source:

This line starts an edit of a file. It loads the file, parses it, converts to a python representation (based on model) and returns a List instance to the user to work with. Since Mod is implemented as a context manager, as soon as with statement’s scope is closed (i.e. when all of the operations defined in this block are completed), it will automatically format the source back to ndf code and write the file out to the destination mod. If you are just tinkering with the code and don’t want to write data out on each test run, you can disable it by adding the following argument:

with mod.edit(r"GameData\Generated\Gameplay\Gfx\Ammunition.ndf", False) as source:

Get and Set Values#

        obj = obj_row.value

        # skip anything that is not of this type
        if obj.type != "TAmmunitionDescriptor": 
            continue

        # NOTE EMBEDDED QUOTES IN COMPARED STRING!
        if obj.by_member('Caliber').value != "'XUTVWWNOTF'": 
            continue

This code skips anything but object with specified caliber. 2 things to note here:

  1. List, Object and Template all store their type inside of their own type attribute instead of row’s one. This issue is covered in detail in typing ambiguity section.

  2. Literal on the right side of the last comparison has 2 sets of quotes. Since ndf-parse stores any trivial types and expressions as strings, we somehow need to keep actual ndf strings distinguishable from say a float value stored as a string literal. For that reason we ebmed actual string quotation within the string.

Finding Items#

There is a walk() function that is designed to help you find stuff around trees. It is recursive so it allows to search through nested objects. Here is an example of how to use it:

import ndf_parse as ndf

mod = ndf.Mod(PATH_TO_SRC_MOD, PATH_TO_DEST_MOD)  # remember to set your paths!

def has_metre(item)->bool:
    # make sure you type check!
    if isinstance(item, ndf.model.MemberRow):
        if isinstance(item.value, str) and 'Metre' in item.value:
            return True
    return False

with mod.edit(r"GameData\Generated\Gameplay\Gfx\Ammunition.ndf", True) as source:
    for metre_row in ndf.walk(source, has_metre):
        print(metre_row)
        ndf.printer.print(metre_row)

This example prints out all items that have an expression including a Metre word. Note that it uses a custom function to check for matches. It is your responsibility to write one that fits your case and to make sure you typecheck.

Generating New NDF items#

To ease the process of making new items there is a convenience function expression(). It takes a string with an ndf statement and returns a dict with the desired object and additional arguments (if the expression was a member/visibility/assignment statement). Example:

import ndf_parse as ndf

mod = ndf.Mod(PATH_TO_SRC_MOD, PATH_TO_DEST_MOD)  # remember to set your paths!
mod.check_if_src_is_newer()

expr = """
NewNS is SomeType(
    Radius = 12 * Metre
    SomeParam = 'A string'
)
"""
# test expression conversion to get a hang of how it works
test_item_dict = ndf.expression(expr)
print(test_item_dict)  # inspect what gets returned

with mod.edit(r"GameData\Generated\Gameplay\Gfx\Ammunition.ndf") as source:
    

    for obj_row in source:
        obj = obj_row.value
        if obj.type != "TAmmunitionDescriptor": 
            continue
        if obj.by_member('Caliber').value != "'XUTVWWNOTF'": 
            continue
        
        # replacing caliber string with an object definition
        new_item_dict = ndf.expression(expr)  # note that we reparse it each time
        obj.by_member('Caliber').edit(**new_item_dict)  # deconstruct dict into args
        # you can edit the added code snippet the same way you edit the tree
        obj.by_member('Caliber').value.by_member('Radius').value += ' + 5'

You should get an output that is an unformatted equivalent to this statement:

{
    'value': [  # DeclarationsList type objects are printed as a python list
        Object[0](
            member='Radius',
            type=None,
            visibility=None,
            namespace=None,
            value='12 * Metre'
        ),
        Object[1](
            member='SomeParam',
            type=None,
            visibility=None,
            namespace=None,
            value="'A string'"
        )
    ],
    'namespace': 'NewNS'
}

Now also check your generated Ammunition.ndf, it should have objects of type SomeType instead of 'XUTVWWNOTF'. It also has it’s Radius modified after replacement. Things worth noting in this example:

  1. We regenerate the expression instead of making it once and pasting for each match. That is necessary because of referencing restriction.

  2. We have multiple new declarations with the same new namespace (“NewNS”). This most likely would not compile but this tool does not enforce language restrictions.

Printing an ndf Code Out#

If you want to print data out (for debugging purposes or whatever), you can do the following:

import ndf_parse as ndf

data = """Obj1 is Type1(
    member1 = Obj2 is Type1(
        member1 = nil
    )
)"""

source = ndf.convert(data)  # manually convert data instead of Mod magic
obj_view = source[0]

print("Complete assignment statement (printing the whole row):")
ndf.printer.print(obj_view)
print("Object declaration only (row's value only):")
ndf.printer.print(obj_view.value)

This code should print out the following:

Complete assignment statement (printing the whole row):
Obj1 is Type1
(
    member1 = Obj2 is Type1
    (
        member1 = nil
    )
)
Object declaration only (row's value only):
Type1
(
    member1 = Obj2 is Type1
    (
        member1 = nil
    )
)

There are 2 other functions you might find useful:

General Recommendations and Caveats#

Errors Suppression#

Avoid using try clause or any other silently failing operations. If Eugen renames or moves objects or members that you’re editing - it’s in your best interest to let the script fail instead of silently ignoring missing member or namespace. That way you will know 100% something has changed in the source code and needs fixing instead of bashing your head over a compiled mod that doesn’t do what you expect from it.

For that reason some functions use strict argument with True by default that forces them to fail if anything is off. Don’t turn those off unless you really know that it won’t hurt you in the long run.

Nested Edits#

Avoid nesting with mod.edit(...) inside of another with mod.edit(...) if they both access the same source file. First clause will build an independent tree from pristine source mode. Second one will build another independent tree from pristine source mode. When second clause ends, your file gets written out with all the changes you made in the second clause. But your first tree still holds data from original unedited tree. As soon as it gets written out, it will overwrite anything you did in the second clause.

Syntax Checking Strictness#

tree-sitter-ndf parser is not a language server so it will allow for some not-quite-correct expressions. It will only catch the most bogus syntax errors while will let through things like excessive commas, multiple unnamed definitions, clashing namespaces and member definitions at the root level. You can read more on this in tree-sitter-ndf’s README.md.

Path Relativeness#

By default python interprets relative paths relative to where the program was started. If for example you have your script in C:\Users\User\Scripts\mod.py but run your terminal from C:\, your script will interpret all relative paths relative to C:\. If you want your script to always internert paths relative to itself, you can add these 2 lines at the beginning of your srcipt:

import os
os.chdir(os.path.dirname(__file__))

This rule however is not applicable to Mod.edit(), Mod.parse_src() and Mod.parse_dst(). These methods operate relative to Mod.mod_src (for Mod.edit() and Mod.parse_src()) and Mod.mod_dst (for Mod.parse_dst(), just make sure you generated some data there before trying to access it).

Typing Ambiguity#

Ndf manual is not very clear on it’s typing annotation rules. Consider the following example:

MemberTyping is TObject(  // object is of type TObject
   // member is of type string
   StringMember : string = "Some text"

   // case similar to one in the manual, we have both member and namespace names
   ObjectMember = InnerNamespace is TObject( ... )

   // syntax allows for this in theory
   WtfMember : MType = CanWeDoThis is TObject( ... )
   AnotherOne : RGBA = RGBA[0, 0, 0, 1]
)

Since there are no clear instructions on wheter this is possible and syntax rules don’t seem to prohibit such declaration, I had to opt for a cursed solution - model.Template, model.Object and model.List have a type parameter that stores their mandatory type declaration (TObject in this example). So for these specific objects don’t rely on row’s type parameter. For everything else Row’s type is the way to go.

No Referencing#

model.DeclarationsList, model.DeclListRow and their subclasses don’t implement copying. If you get a value from one row (or a row itself) and put it into another row then they both will refer to the same object in memory. This will lead to unexpected edits at best (when modifying one item, other will sync with it) and infinite recursions on printing at worst (if some object happens to become a child of itself, irrespective of depth of such hierarchy). This can be fixed in future by implementing deep copying on assignment.