Tools for handling resource code

Forum for alternative clients, mods & discussions on the same.

Tools for handling resource code

Postby loftar » Sun Jun 05, 2022 11:14 pm

Resources containing code are, as many are already acutely aware, a source of constant frustration for client modders. I've already talked about it elsewhere, but there are numerous reasons for me to use resource code, and while I'm not going to part with it, I have lately introduced a couple of tools for client modders that I hope should reduce the friction. There is somewhat detailed documentation/ available in the client source tree, so please refer to it for all details. In this post, I'll mainly describe what is being introduced.

The changes have not yet been merged to the client's master branch, but are available in the "res-override" branch for you to preview. Unless there are any damning objections, I'll merge them to mainline shortly, and you can merge them from there.

Fundamentally, not much has changed, but there are three main additions to client (the third one of which may be slightly controversial).

1. A tool to fetch resource source code
Since a while back, resources with code also contain their source code, but this has never been documented or supported in any way. The client now contains a command-line tool that can be invoked to fetch the source code of a resource. Again, please see the documentation linked above for the details, but briefly, you can run something like this to fetch the source code for a resource:

Code: Select all
java -cp bin/hafen.jar haven.Resource get-code ui/tt/name


That will put the source code in the src directory along with all the rest of the client source code, in the subdirectory appropriate for the package name the code uses, and as such, that code will compile along with the rest of the source code. Since the fetched code lives along with the rest of the client code (in the same classpath, as far as Java and JavaC are concerned), there should be no issues referring back and forth between it and the ordinary code without needing any reflection.

Note that there may exist some resources that don't contain source code (in which case the above command will complain). Such resources are simply historical artifacts of the fact that I may not have recompiled them since I added the ability for resources to contain their source code, rather than being intentional. If you happen to come across any, just report them and I'll recompile them.

2. A tool to check for updates
When you use the get-code command to fetch resource code, it will annotate the classes thus fetched with a FromResource annotation, containing the name and version of the resource it was fetched from. Along with get-code, there is also a find-updates command that scans all source files for such annotations, and checks the corresponding resources for newer versions. It would usually be invoked without any arguments:

Code: Select all
java -cp bin/hafen.jar haven.Resource find-updates


It will simply list the resources which have newer versions, without actually doing any updates. That is simply so that you can check for updates without worrying about anything being overwritten. Just use get-code again to actually fetch the updates.

3. A mechanism to automatically let newer resources override local code
The resource classloader also uses the annotations to make it so that, if there is a newer version of a resource than that which the local code was compiled from, the code from the new resource overrides the local code. I realize this may be somewhat controversial, but the idea is to make custom clients more crash-resistant, if you will bear with me. The reason is because resource updates are sometimes incompatible with previously fetched local code, and as we've seen many times already, if the client is left un-updated, it may simply crash. Allowing newer resources to override local code should allow the client to continue running, albeit with degraded functionality, until its author will come around to update it. I do realize that there are also many resource updates that are not incompatible and will thus degrade client functionality unnecessarily, but I should think that occasional degraded functionality is better than outright crashes when incompatible changes do come around.

It should be noted that this override mechanism will also override local classes that simply don't have the FromResource annotation, as would obviously be the case with current extracted code that doesn't come via get-code. If you don't want to, you don't have to use get-code just to add the FromResource annotation; whenever this happens, the client will emit a warning (to standard error) which contains the name and version of the resource that is being used to override local code, so you can simply use the information from that warning to add annotations yourself.

Again, I do realize that this may be a bit controversial, and if you really hate it, there are ways to turn it off, both globally and more selectively (also desribed in the above linked documentation), but please consider that keeping this behavior probably makes your client more crash-resistant in the face of incompatible resource changes.


Also, inb4
How 'bout you just put all the code in Git and spare us the pleasure of your custom tools, ya doofus?

There are many reasons to keep the resource-code system. Some of them are purely for my own convenience, but the idea is that it's also for your sakes. If resource-code were simply part of the rest of the client code, then any time we push an update that incorporates new code, then all clients would just crash all around until they're updated. If you stop and think about it a wee bit, I think even you may come to agree that it's better that I can push some simple tooltip or harmless sprite or something without every custom client breaking until their authors get around to update them.

It should also be noted, that if you just want to use Git to manage merges and conflicts with resource code, there is a way to do so. It is described in the above linked documentation, so refer to that in case you're interested.

At the very least, I hope these tools will lessen the frustration a bit. I'm sure you'll let me know if you have any opinions about them.
"Object-oriented design is the roman numerals of computing." -- Rob Pike
User avatar
loftar
 
Posts: 8926
Joined: Fri Apr 03, 2009 7:05 am

Re: Tools for handling resource code

Postby strpk0 » Sun Jun 05, 2022 11:30 pm

3. A mechanism to automatically let newer resources override local code
The resource classloader also uses the annotations to make it so that, if there is a newer version of a resource than that which the local code was compiled from, the code from the new resource overrides the local code. I realize this may be somewhat controversial, but the idea is to make custom clients more crash-resistant, if you will bear with me. The reason is because resource updates are sometimes incompatible with previously fetched local code, and as we've seen many times already, if the client is left un-updated, it may simply crash. Allowing newer resources to override local code should allow the client to continue running, albeit with degraded functionality, until its author will come around to update it. I do realize that there are also many resource updates that are not incompatible and will thus degrade client functionality unnecessarily, but I should think that occasional degraded functionality is better than outright crashes when incompatible changes do come around.


No. Dear god no. Please.
This doesn't solve anything, only creates more maintainability problems for custom client makers. At the very least we can currently keep track of res loaded code updates and merge them (or not) at our own discretion. Letting the client overwrite classes every time you guys decide to change something is just going to lead to more crashes and overall custom client bugginess.

You're heavily underestimating how much work goes into res-loaded code if you think its as simple as "just merge it lol" whenever you guys decide to change something. Sometimes not merging at all and just fixing the incompatibilities is the only feasible solution. Having things randomly break instead of on a per-game-update basis is just a much worse version of what currently is happening.

EDIT: Didn't notice it was possible to turn it off, nevermind. I'm still not very sure this will fix custom client crashes considering how incompatible the modifications to res loaded code usually are with default client code.
Granger wrote:Fuck off, please go grow yourself some decency.

Image
User avatar
strpk0
 
Posts: 1131
Joined: Sat Sep 03, 2011 11:44 pm

Re: Tools for handling resource code

Postby loftar » Sun Jun 05, 2022 11:38 pm

strpk0 wrote:EDIT: Didn't notice it was possible to turn it off, nevermind. I'm still not very sure this will fix custom client crashes considering how incompatible the modifications to res loaded code usually are with default client code.

How so, though? The way I see it, if newer resource code is used in preference to local code, that shouldn't cause anything to crash in itself, while it should prevent out-of-date code from crashing on newer interfaces or protocol formats, but please inform me if there's something I'm missing.

strpk0 wrote:You're heavily underestimating how much work goes into res-loaded code if you think its as simple as "just merge it lol" whenever you guys decide to change something. Sometimes not merging at all and just fixing the incompatibilities is the only feasible solution.

Also that wasn't my implication at all; the only intention here is to let newer resource versions override local code until said local code can be updated, whether that involves simple merging or more complex fixes, or otherwise. Really, my only intention is to avoid outright crashes that make custom clients unusable until they are fixed. In the end, it's completely up to you to decide whether to use it, but the idea was to improve the situation for custom clients, not to make it worse, so I'd urge you to consider it.
"Object-oriented design is the roman numerals of computing." -- Rob Pike
User avatar
loftar
 
Posts: 8926
Joined: Fri Apr 03, 2009 7:05 am

Re: Tools for handling resource code

Postby VDZ » Mon Jun 06, 2022 12:50 am

loftar wrote:
strpk0 wrote:EDIT: Didn't notice it was possible to turn it off, nevermind. I'm still not very sure this will fix custom client crashes considering how incompatible the modifications to res loaded code usually are with default client code.

How so, though? The way I see it, if newer resource code is used in preference to local code, that shouldn't cause anything to crash in itself, while it should prevent out-of-date code from crashing on newer interfaces or protocol formats, but please inform me if there's something I'm missing.

I haven't touched client code since Legacy myself, but consider the scenario where custom code sets a variable, that code gets overwritten and thus not executed, then other custom code which is still in uses that variable and does not find it in the expected state (worst case: variable is null and assumed non-null).

loftar wrote:Note that there may exist some resources that don't contain source code (in which case the above command will complain). Such resources are simply historical artifacts of the fact that I may not have recompiled them since I added the ability for resources to contain their source code, rather than being intentional. If you happen to come across any, just report them and I'll recompile them.

Why not just run get-code on every resource and recompile whatever resources it complains about? Seems more efficient than handling reports case-by-case.
User avatar
VDZ
 
Posts: 2660
Joined: Sun Jul 17, 2011 2:27 am

Re: Tools for handling resource code

Postby loftar » Mon Jun 06, 2022 1:03 am

VDZ wrote:I haven't touched client code since Legacy myself, but consider the scenario where custom code sets a variable, that code gets overwritten and thus not executed, then other custom code which is still in uses that variable and does not find it in the expected state (worst case: variable is null and assumed non-null).

Since all the code we're talking about is resource-triggered code, I'd expect any code that interacts with it from "the other side" to have to handle the situation of that code not having been triggered (yet, or at all during a session) anyway. I think generally speaking, a resource being overridden should be the same to code that looks for it as that resource simply not having been used.
"Object-oriented design is the roman numerals of computing." -- Rob Pike
User avatar
loftar
 
Posts: 8926
Joined: Fri Apr 03, 2009 7:05 am

Re: Tools for handling resource code

Postby strpk0 » Mon Jun 06, 2022 1:30 am

My previous post aside, the other tools in this post are a great step in the right direction. Not needing 3rd party tools to work on res-loaded code is definitely appreciated.
Granger wrote:Fuck off, please go grow yourself some decency.

Image
User avatar
strpk0
 
Posts: 1131
Joined: Sat Sep 03, 2011 11:44 pm

Re: Tools for handling resource code

Postby shubla » Mon Jun 06, 2022 8:07 pm

Why not just recompile all resources? Fear the wasted bandwidth to send the new res files? We don't live in the age of ADSL anymore!
Image
I'm not sure that I have a strong argument against sketch colors - Jorb, November 2019
http://i.imgur.com/CRrirds.png?1
Join the moderated unofficial discord for the game! https://discord.gg/2TAbGj2
Purus Pasta, The Best Client
User avatar
shubla
 
Posts: 13043
Joined: Sun Nov 03, 2013 11:26 am
Location: Finland

Re: Tools for handling resource code

Postby telum12 » Wed Jun 08, 2022 3:25 pm

loftar wrote:
VDZ wrote:I haven't touched client code since Legacy myself, but consider the scenario where custom code sets a variable, that code gets overwritten and thus not executed, then other custom code which is still in uses that variable and does not find it in the expected state (worst case: variable is null and assumed non-null).

Since all the code we're talking about is resource-triggered code, I'd expect any code that interacts with it from "the other side" to have to handle the situation of that code not having been triggered (yet, or at all during a session) anyway. I think generally speaking, a resource being overridden should be the same to code that looks for it as that resource simply not having been used.


The reason that it is (currently) problematic is that this was not available before, so we have generally not designed res class modifications defensively. Since updated res classes would break regardless, there was no need to make custom clients compatible with the non-edited res classes. Being able to turn off this feature will give a much needed respite while changing this approach. Regardless, having this option is probably going to be helpful, since the public clients are somewhat over-reliant on a few people having time to merge res files (assuming any outstanding design issues are resolved).

I do expect some confused "bug" posts when some features unexpectedly vanish though, lmao.

It's fantastic to see public clients seeing some love, since the QoL they provide to many players is often substantial.
MagicManICT wrote:To me, being called a pedo is exactly like being called gay.

Jalpha wrote:She must have been in heat bro. She was literally fanging for it. Literally posting repeatedly in chat, in all caps "DO IT! POST YOUR DICK! THERE'S NO WAY IT'S 7 INCHES!"

How could any hot-blooded male deny such a request under the circumstances.
User avatar
telum12
 
Posts: 426
Joined: Mon Mar 12, 2012 10:36 pm

Re: Tools for handling resource code

Postby loftar » Wed Jun 08, 2022 6:22 pm

telum12 wrote:The reason that it is (currently) problematic is that this was not available before, so we have generally not designed res class modifications defensively. Since updated res classes would break regardless, there was no need to make custom clients compatible with the non-edited res classes.

I have no problem understanding that would be the mindset, I just can't really imagine a way in which it would cause a real-life problem. The way I imagine it working, a locally modified resource class being overridden with the code actually in the resource would just basically look the same to the client as if the server hadn't asked for the resource to be used to begin with, and that is a situation that I imagine that all custom clients need to deal with regardless. I guess I'm just kind of curious what an example would be of something that would cause a crash or other sort of serious problem right now.

It should be noted, in case there is any confusion in that regard, that a resource classloader overriding a locally modified class doesn't actually cause anything to happen with that class in the application classloader. All the other classes loaded via the application classloader would still see it and be able to interact with it in the expected way, and the overridden class that comes via the resource classloader would simply look like a separate and different class (that just happens to have the same name, but a different classloader).

telum12 wrote:I do expect some confused "bug" posts when some features unexpectedly vanish though, lmao.

Lol yeah, I know. It's not exactly optimal, but I reckon it's still better than crashing. I guess a client could show a warning that resources are being overridden (the default code already does, but only to standard error), and/or even have an option to control the override behavior, but that's of course entirely your choice to make.
"Object-oriented design is the roman numerals of computing." -- Rob Pike
User avatar
loftar
 
Posts: 8926
Joined: Fri Apr 03, 2009 7:05 am

Re: Tools for handling resource code

Postby iamahh » Thu Jun 09, 2022 2:12 pm

seems running on IntelliJ terminal requires some authentication?

Code: Select all
java -jar bin/hafen.jar haven.Resource get-code ui/tt/level
Exception in thread "Haven main thread" java.lang.RuntimeException: No explicit or saved username
        at haven.MainFrame.connect(MainFrame.java:231)
        at haven.MainFrame.main2(MainFrame.java:385)
        at haven.MainFrame.lambda$main$0(MainFrame.java:422)
        at java.base/java.lang.Thread.run(Thread.java:829)


Code: Select all
java -jar bin/hafen.jar haven.Resource find-updates
Exception in thread "Haven main thread" java.lang.RuntimeException: No explicit or saved username
        at haven.MainFrame.connect(MainFrame.java:231)
        at haven.MainFrame.main2(MainFrame.java:385)
        at haven.MainFrame.lambda$main$0(MainFrame.java:422)
        at java.base/java.lang.Thread.run(Thread.java:829)
iamahh
 
Posts: 1810
Joined: Sat Dec 12, 2015 8:23 pm

Next

Return to The Wizards' Tower

Who is online

Users browsing this forum: Naylok, Yandex [Bot] and 12 guests