![[sticky entry]](http://www.love4eva.net/img/silk/entry/sticky_entry.png){kind=small}
lol | bsky | sprlv | ao3 | ttrpgs | vidya
Entry tags:
toxic yuri optimization: building lighter-weight executable files for your game releases
so at some point while trying to read as many entries as possible of last year's Toxic Yuri VN Jam I fucking lost it because the thing about the most common engine is that it maybe—if you use next to no assets—might be like a couple hundred megs and on average ends up around 300mb to 1gb. This is really annoying when you are running out of disk space and there are a jillionty entries you really really wanna read (i have most of them on a flash drive now.)
so since then i've been idly thinking about how to build a better workflow for making VNs that doesn't compile to a huge fuckoff executable in prep for this year's TYVNJ. i think i have figured it out, mostly, although some of the component steps are only for insane people (me). anyway here's some notes about it.
aka what you do if you hate yourself (it's joke. i love myself i am a brilliant brain genius who is so good at programming)
ok. so. cross-compilation has a number of solutions, some of which are good and some of which suck. the easiest way to ship cross-platform application builds is to make them in html/css/js and package them with electron, which functionally wraps them in an embedded chrome browser. i do like to work in html/css/js because it's basically a native language to me at this point and i'm pretty sure i can make it do literally anything. the problem is that electron kind of locks it at a minimum of ~150mb/executable since it has to package a full chrome build in there, which sucks since anything you add balloons the size from there. ren'py builds also tend to be pretty chonky although i have less insight into the compiler situation there since it's built off of pygame. in my case, i was building off of videotome adv because the aesthetic and sensibilities of the engine really fit the game imo although what i've done to it is uhhhh. well. possibly this is a ship of theseus situation
if you want to ship something lighter, you do have options—particularly using the webview library, which shops out the browser interface to libraries easily available for the platform. however this means it uses platform-specific shit for each thing, which means the windows libs aren't installable on mac and so on. this is understandably a pain in the ass to cross-compile for unless you have all the types of computer or full virtual machines on hand. (if you're just doing it in raw graphics code of whatever lang you're using then congrats, you can just use whatever cross-compiler is available. i hear zig is really easy to cross-compile ftr)
i hate to say that github is the solution but github is the solution. most other sites that have some kind of automatic runner situation don't necessarily offer the ability to compile on windows or macos, and self-hosting an action runner definitely doesn't do that, so. github it is. believe me, i tried figuring out basically everything else.
so first off you have to have a github repo. i was storing my work on git.gay (great place btw) so I added an extra remote with
mine looks like the below. it could probably be better in some ways but this is my first time using actions so "good enough" was like. fine
ok that's a lot so i'm going to break it down into parts.
the first block is basically saying: when someone pushes to main, it should kick off this process.
the strategy matrix defines the operating systems we're building on; you can see i have ubuntu, macos, and windows on latest versions. if you're using a different application compiler (i'm using electrobun; there's also zero-native, tauri, a number of others, pick your poison and what backend you want if any) then you might want to have different variables, idk, i don't know your life. this tells it to run the process on all of these virtual machines so that it'll produce a native build for that type of OS. which is what we want! yay. the folder base you should keep noted though because regardless of what you're doing that's probably a secret tool that will help you later.
so first there's that
As you can see, each of these blocks has an
I'm using bun as my package manager and language runtime, so I have to have a step installing that also. if you're using something else you might have installed the language in the OS package install step prior, or you might be running the equivalent for deno or node or composer or something here.
then after you've got your dependencies you run your application build step. presumably your app builder will have given you a build script to place in your package.json file (or you know. something else but if you're building with something else you probably get that already). i also make a release folder to fuck around with zipping some files but that's kind of a personal taste thing, since you can do that anywhere really. so anyway after this step there will be built executables on each machine for that OS!
so now here's the last bit where we have to get the files to a place where we can touch them. i have an account with bunny.net (full disclosure that's an affiliate link. i highly recommend them though and they have a lot of good services on the cheap. like it costs me a buck a month) with a storage zone set up so i can beam files to it. for that i got an API key from them which i put in my repo secrets under repository settings > secrets. (make sure you get the base URL right for your storage location; i forgot to put the la prefix on and it did Not work.) i also put the exact folder path and my storage zone name in my secrets because I don't want randos knowing them.
so there are separate zip steps for mac and linux because the mac version needs to be zipping a file with the .app suffix (otherwise operating systems interpret it as a folder because a .app is technically a kind of folder, actually); otherwise i could do them the same (this is very annoying but we stay silly). the windows file does not need to be zipped
then what i am doing is beaming the files to the CDN storage zone via the HTTP API. if you have some other CDN situ it probably has its own instructions for its endpoint; i think there's also some actions in the "marketplace" on gh for uploading to (S)FTP but i haven't tried them so can't speak to it. (there's probably other things you can do with it, too but at that point i was like GOOD ENOUGH)
and, presuming you don't run into any compilation errors... you should have several built executable files in your storage to distribute. for reference, the compiled executables for our project clocked in around 50-60MB. no i am not missing any zeroes. this is with a full soundtrack and a lot of art; if you're doing something comparatively simpler like packaging a twine game, it'll be really small, probably
this was obviously a lot and you cannot even begin to imagine how long this took me to figure out but i figure i might as well document it for the next person who tries to do something deranged like this.
good lord. anyway our entry was in fact only semi-functional at cutoff time. but i maintain that it was not because of my deranged custom engine stuff and more due to Unavoidable Life Events
so since then i've been idly thinking about how to build a better workflow for making VNs that doesn't compile to a huge fuckoff executable in prep for this year's TYVNJ. i think i have figured it out, mostly, although some of the component steps are only for insane people (me). anyway here's some notes about it.
the easy stuff
- Turn all your images into webps if possible and all your audio into oggs. I'm serious. I know a lot of people hate webps but support in applications is increasing and they have wildly good compression and can be animated. i use this thing to turn videos into animated webps and, well... ezgif.com's audio tools (?????) to convert whatever to .ogg. you can also do this with ffmpeg's cli on your computer but whatever the default settings are on ezgif, they're pretty good
- also if you've got svgs anywhere here's an optimizer
- if you're doing stuff in html/css like i am, optimize your webfonts into woff2
- vite is a pretty straightforward bundler/combiner/optimizer/asset workflow tool for web projects that has plugins for dealing with most imaginable file types; it can be as simple or as complicated as you want and also it means you can split your stuff into as many files as you want for parseability but not have to make a bazillion resource calls in the browser
executable cross-compilation for html games is the fucking worst but we do it anyway
aka what you do if you hate yourself (it's joke. i love myself i am a brilliant brain genius who is so good at programming)
ok. so. cross-compilation has a number of solutions, some of which are good and some of which suck. the easiest way to ship cross-platform application builds is to make them in html/css/js and package them with electron, which functionally wraps them in an embedded chrome browser. i do like to work in html/css/js because it's basically a native language to me at this point and i'm pretty sure i can make it do literally anything. the problem is that electron kind of locks it at a minimum of ~150mb/executable since it has to package a full chrome build in there, which sucks since anything you add balloons the size from there. ren'py builds also tend to be pretty chonky although i have less insight into the compiler situation there since it's built off of pygame. in my case, i was building off of videotome adv because the aesthetic and sensibilities of the engine really fit the game imo although what i've done to it is uhhhh. well. possibly this is a ship of theseus situation
if you want to ship something lighter, you do have options—particularly using the webview library, which shops out the browser interface to libraries easily available for the platform. however this means it uses platform-specific shit for each thing, which means the windows libs aren't installable on mac and so on. this is understandably a pain in the ass to cross-compile for unless you have all the types of computer or full virtual machines on hand. (if you're just doing it in raw graphics code of whatever lang you're using then congrats, you can just use whatever cross-compiler is available. i hear zig is really easy to cross-compile ftr)
i hate to say that github is the solution but github is the solution. most other sites that have some kind of automatic runner situation don't necessarily offer the ability to compile on windows or macos, and self-hosting an action runner definitely doesn't do that, so. github it is. believe me, i tried figuring out basically everything else.
so first off you have to have a github repo. i was storing my work on git.gay (great place btw) so I added an extra remote with
git remote add gh [url] so I could push to both. iirc actions are enabled by default on new repos, so you should be able to just go in and make one under that tab.mine looks like the below. it could probably be better in some ways but this is my first time using actions so "good enough" was like. fine
name: cross-compilation hell
on:
push:
branches: [main]
jobs:
build:
strategy:
matrix:
include:
- os: ubuntu-latest
artifact_name: HOME_MOVIES_8-3-2002
artifact_ext: ''
buildfolder: 'stable-linux-x64'
osname: 'linux'
folderbase: /home/runner/work
- os: macos-latest
artifact_name: HOME_MOVIES_8-3-2002
artifact_ext: '.app'
buildfolder: 'stable-macos-arm64'
osname: 'macos'
folderbase: /Users/runner/work
- os: windows-latest
artifact_name: HOME_MOVIES_8-3-2002
artifact_ext: '.exe'
buildfolder: 'stable-win-x64'
osname: 'windows'
folderbase: /d/a
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v7
# Install platform-specific dependencies
- name: Install Linux dependencies
if: runner.os == 'Linux'
run: |
sudo apt-get update
sudo apt-get install -y libgtk-3-dev libwebkit2gtk-4.1-dev unzip zip
- name: Install macOS dependencies
if: runner.os == 'macOS'
run: |
brew install pkg-config
- name: Install Windows dependencies
if: runner.os == 'Windows'
run: |
choco install visualstudio2022buildtools -y
- name: Setup Bun
uses: oven-sh/setup-bun@v2.2.0
with:
bun-version: latest
- name: Install Packages
run: bun --bun i
- name: Build application
run: bun --bun run eb:build:release
- name: Make Releases Folder
shell: bash
run: |
mkdir -p release
- name: Zip Files for Upload
if: runner.os == 'macOS'
uses: TheDoctor0/zip-release@0.7.6
with:
filename: HOME_MOVIES_8-3-2002-${{ matrix.osname }}.zip
directory: ${{ matrix.folderbase }}/tyvnj2/tyvnj2/release
path: ../build/${{ matrix.buildfolder }}/HOME_MOVIES_8-3-2002.app
custom: -r
- name: Zip Files for Upload
if: runner.os == 'Linux'
uses: TheDoctor0/zip-release@0.7.6
with:
filename: HOME_MOVIES_8-3-2002-${{ matrix.osname }}.zip
directory: ${{ matrix.folderbase }}/tyvnj2/tyvnj2/release
path: ../build/${{ matrix.buildfolder }}/HOME_MOVIES_8-3-2002
custom: -r
- name: CURL Upload Windows Build to Bunny Storage
if: runner.os == 'Windows'
shell: bash
run: |
curl -X PUT https://la.storage.bunnycdn.com/${{ secrets.BUNNY_STORAGE_ZONE }}/${{ secrets.BUNNY_FOLDER_PATH }}/HOME_MOVIES_8-3-2002-${{ matrix.osname }}.exe \
-H "AccessKey: ${{ secrets.BUNNY_UPLOAD }}" \
-H "Content-Type: application/zip" \
--upload-file ${{ matrix.folderbase }}/tyvnj2/tyvnj2/build/${{ matrix.buildfolder }}/HOME_MOVIES_8-3-2002-Setup.exe
- name: CURL Upload Build to Bunny Storage
if: runner.os == 'Linux' || runner.os == 'macOS'
shell: bash
run: |
curl -X PUT https://la.storage.bunnycdn.com/${{ secrets.BUNNY_STORAGE_ZONE }}/${{ secrets.BUNNY_FOLDER_PATH }}/HOME_MOVIES_8-3-2002-${{ matrix.osname }}.zip \
-H "AccessKey: ${{ secrets.BUNNY_UPLOAD }}" \
-H "Content-Type: application/zip" \
--upload-file ${{ matrix.folderbase }}/tyvnj2/tyvnj2/release/HOME_MOVIES_8-3-2002-${{ matrix.osname }}.zip
- name: CURL Upload DMG to Bunny Storage
if: runner.os == 'macOS'
shell: bash
run: |
curl -X PUT https://la.storage.bunnycdn.com/${{ secrets.BUNNY_STORAGE_ZONE }}/${{ secrets.BUNNY_FOLDER_PATH }}/HOME_MOVIES_8-3-2002-${{ matrix.osname }}.dmg \
-H "AccessKey: ${{ secrets.BUNNY_UPLOAD }}" \
-H "Content-Type: application/zip" \
--upload-file ${{ matrix.folderbase }}/tyvnj2/tyvnj2/artifacts/stable-macos-arm64-HOME_MOVIES_8-3-2002.dmgok that's a lot so i'm going to break it down into parts.
name: cross-compilation hell
on:
push:
branches: [main]the first block is basically saying: when someone pushes to main, it should kick off this process.
jobs:
build:
strategy:
matrix:
include:
- os: ubuntu-latest
artifact_name: HOME_MOVIES_8-3-2002
artifact_ext: ''
buildfolder: 'stable-linux-x64'
osname: 'linux'
folderbase: /home/runner/work
- os: macos-latest
artifact_name: HOME_MOVIES_8-3-2002
artifact_ext: '.app'
buildfolder: 'stable-macos-arm64'
osname: 'macos'
folderbase: /Users/runner/work
- os: windows-latest
artifact_name: HOME_MOVIES_8-3-2002
artifact_ext: '.exe'
buildfolder: 'stable-win-x64'
osname: 'windows'
folderbase: /d/a
runs-on: ${{ matrix.os }}the strategy matrix defines the operating systems we're building on; you can see i have ubuntu, macos, and windows on latest versions. if you're using a different application compiler (i'm using electrobun; there's also zero-native, tauri, a number of others, pick your poison and what backend you want if any) then you might want to have different variables, idk, i don't know your life. this tells it to run the process on all of these virtual machines so that it'll produce a native build for that type of OS. which is what we want! yay. the folder base you should keep noted though because regardless of what you're doing that's probably a secret tool that will help you later.
steps:
- uses: actions/checkout@v7
- name: Get System Info
uses: lexbritvin/os-info-action@v1
id: systeminfo
# Install platform-specific dependencies
- name: Install Linux dependencies
if: runner.os == 'Linux'
run: |
sudo apt-get update
sudo apt-get install -y libgtk-3-dev libwebkit2gtk-4.1-dev unzip zip
- name: Install macOS dependencies
if: runner.os == 'macOS'
run: |
brew install pkg-config
- name: Install Windows dependencies
if: runner.os == 'Windows'
run: |
choco install visualstudio2022buildtools -y
so first there's that
uses step which just imports the action utilities from github. you should have it but you don't need to worry about itAs you can see, each of these blocks has an
if property, which as you might guess tells it what conditions to run under. We're using it here to run a separate package install process for each OS, since they all have different package managers and need different dependencies. this is probably all you need and if you need any additional packages the build process errors will tell you. - name: Setup Bun
uses: oven-sh/setup-bun@v2.2.0
with:
bun-version: latest
- name: Install Packages
run: bun --bun i
- name: Build application
run: bun --bun run eb:build:release
- name: Make Releases Folder
shell: bash
run: |
mkdir -p releaseI'm using bun as my package manager and language runtime, so I have to have a step installing that also. if you're using something else you might have installed the language in the OS package install step prior, or you might be running the equivalent for deno or node or composer or something here.
bun i is the dependency installation for the project; if you're using a different language, substitute that package manager's install script (e.g. yarn, deno install, pnpm install, composer install, whatever).then after you've got your dependencies you run your application build step. presumably your app builder will have given you a build script to place in your package.json file (or you know. something else but if you're building with something else you probably get that already). i also make a release folder to fuck around with zipping some files but that's kind of a personal taste thing, since you can do that anywhere really. so anyway after this step there will be built executables on each machine for that OS!
- name: Zip Files for Upload
if: runner.os == 'macOS'
uses: TheDoctor0/zip-release@0.7.6
with:
filename: HOME_MOVIES_8-3-2002-${{ matrix.osname }}.zip
directory: ${{ matrix.folderbase }}/tyvnj2/tyvnj2/release
path: ../build/${{ matrix.buildfolder }}/HOME_MOVIES_8-3-2002.app
custom: -r
- name: Zip Files for Upload
if: runner.os == 'Linux'
uses: TheDoctor0/zip-release@0.7.6
with:
filename: HOME_MOVIES_8-3-2002-${{ matrix.osname }}.zip
directory: ${{ matrix.folderbase }}/tyvnj2/tyvnj2/release
path: ../build/${{ matrix.buildfolder }}/HOME_MOVIES_8-3-2002
custom: -r
- name: CURL Upload Windows Build to Bunny Storage
if: runner.os == 'Windows'
shell: bash
run: |
curl -X PUT https://la.storage.bunnycdn.com/${{ secrets.BUNNY_STORAGE_ZONE }}/${{ secrets.BUNNY_FOLDER_PATH }}/HOME_MOVIES_8-3-2002-${{ matrix.osname }}.exe \
-H "AccessKey: ${{ secrets.BUNNY_UPLOAD }}" \
-H "Content-Type: application/vnd.microsoft.portable-executable" \
--upload-file ${{ matrix.folderbase }}/tyvnj2/tyvnj2/build/${{ matrix.buildfolder }}/HOME_MOVIES_8-3-2002-Setup.exe
- name: CURL Upload Build to Bunny Storage
if: runner.os == 'Linux' || runner.os == 'macOS'
shell: bash
run: |
curl -X PUT https://la.storage.bunnycdn.com/${{ secrets.BUNNY_STORAGE_ZONE }}/${{ secrets.BUNNY_FOLDER_PATH }}/HOME_MOVIES_8-3-2002-${{ matrix.osname }}.zip \
-H "AccessKey: ${{ secrets.BUNNY_UPLOAD }}" \
-H "Content-Type: application/zip" \
--upload-file ${{ matrix.folderbase }}/tyvnj2/tyvnj2/release/HOME_MOVIES_8-3-2002-${{ matrix.osname }}.zip
- name: CURL Upload DMG to Bunny Storage
if: runner.os == 'macOS'
shell: bash
run: |
curl -X PUT https://la.storage.bunnycdn.com/${{ secrets.BUNNY_STORAGE_ZONE }}/${{ secrets.BUNNY_FOLDER_PATH }}/HOME_MOVIES_8-3-2002-${{ matrix.osname }}.dmg \
-H "AccessKey: ${{ secrets.BUNNY_UPLOAD }}" \
-H "Content-Type: application/zip" \
--upload-file ${{ matrix.folderbase }}/tyvnj2/tyvnj2/artifacts/stable-macos-arm64-HOME_MOVIES_8-3-2002.dmgso now here's the last bit where we have to get the files to a place where we can touch them. i have an account with bunny.net (full disclosure that's an affiliate link. i highly recommend them though and they have a lot of good services on the cheap. like it costs me a buck a month) with a storage zone set up so i can beam files to it. for that i got an API key from them which i put in my repo secrets under repository settings > secrets. (make sure you get the base URL right for your storage location; i forgot to put the la prefix on and it did Not work.) i also put the exact folder path and my storage zone name in my secrets because I don't want randos knowing them.
so there are separate zip steps for mac and linux because the mac version needs to be zipping a file with the .app suffix (otherwise operating systems interpret it as a folder because a .app is technically a kind of folder, actually); otherwise i could do them the same (this is very annoying but we stay silly). the windows file does not need to be zipped
then what i am doing is beaming the files to the CDN storage zone via the HTTP API. if you have some other CDN situ it probably has its own instructions for its endpoint; i think there's also some actions in the "marketplace" on gh for uploading to (S)FTP but i haven't tried them so can't speak to it. (there's probably other things you can do with it, too but at that point i was like GOOD ENOUGH)
and, presuming you don't run into any compilation errors... you should have several built executable files in your storage to distribute. for reference, the compiled executables for our project clocked in around 50-60MB. no i am not missing any zeroes. this is with a full soundtrack and a lot of art; if you're doing something comparatively simpler like packaging a twine game, it'll be really small, probably
this was obviously a lot and you cannot even begin to imagine how long this took me to figure out but i figure i might as well document it for the next person who tries to do something deranged like this.
good lord. anyway our entry was in fact only semi-functional at cutoff time. but i maintain that it was not because of my deranged custom engine stuff and more due to Unavoidable Life Events
thinking about Posts™
Thinking about what kinds of posts I could make here since I'm bad at remembering to post in slightly longer formats
- reviewing recent supreme court opinions
- poetry book reading notes
- secret sneak peaks into gamedev stuff that no other platform gets to see