Resource Layer Utility

Forum for alternative clients, mods & discussions on the same.

Moderator: Phades

Resource Layer Utility

Postby boshaw » Mon Nov 07, 2011 12:46 am

LayerUtil
For download/instructions see: http://forum.salemthegame.com/viewtopic ... 417#p63185
If you are running x64 you can use the following precompiled package: http://puu.sh/fagnD/be9e069e72.rar

Noted Bugs


Where to get it?
See: http://forum.salemthegame.com/viewtopic ... 417#p63185

What is it?
LayerUtil is a similar utility to theTrav’s havenResUtil, but I took it a step further to the point of where every layer in the resource file is decoded rather than just the image.png. I have tested it on the whole haven-res and decoded then re-encoded with my LayerUtil and used the haven-res with no errors so it does work.

What if you find a bug?
Post in the thread, copy paste any error messages it leaves and i'll try to fix them or i'll try to tell you what you did wrong.

How does it work?
Code: Select all
-rd:
-->check [IN]\meta for skips on .res/.cache files; IF no [IN]\meta it's assumed no skip
-->loop through all files
---->Does this file get skipped?
------->Yes: Ignore continue to next file
------->No: decode it; When we write to this .res/.cache file's `meta' it'll be of value SKIP (1) or if modifier `-ns' it'll be of value NO SKIP (0)

-re:
-->loop through all files
---->Check this file's `meta' to see if it's skipped
------->Yes: Ignore and continue to next file
------->No: Is forced skip modified on? `-fs0 or -fs1'
---------->Yes: add this file's skip value to [OUT]\meta as the forced skip value
------->No: Was it previously defined in [OUT]\meta ?
---------->Yes: add this file's skip value to [OUT]\meta as what the previous skip value found in [OUT]\meta was
------->No: add this file's skip value to [OUT]\meta as to what the current skip_value is; default = SKIP(1), IF modifier `-ns' then skip_value = NOSKIP(0)
------->encode the file
-->loop through any remaining previously defined skip values and rewrite them so they aren't lost.

For info on the skip system can be found @ : viewtopic.php?f=27&t=23458&p=299124#p299124


Important Information
1) '#' as the first character of a line signifies a comment that'll be ignored.
3) for option arguments don't use absolute paths like "C:\...\...\..\",etc it'll error out i can guarantee that.
4) In decoded files when you see something that takes the type `String' that means it'll only take the next string that's 1 line if you want multiple line strings use '\n' char when you want to signal a newline
Example:
Code: Select all
#This is the wrong way to do string types
Apple
Bees


Code: Select all
#this is the correct way to do String types
Apple\nBees


Structure of decoded files
The way I have it setup is that whenever you decode x.res it’ll go and create `out_folder/x.res/’ directory and then each layer that x.res contains is decoded into its own folders and then within those layer folders is where you’ll find .data files, .png files, .class files,etc

What are these `Layers’?
The Resource system in hnh is setup so that each file can contain 12 different layer types and these types can be contained in the .res file multiple times so that if x.res has 2 images it’ll have 2 image layers,etc.
So what layers are there? : { image, tile, neg, anim, tileset, pagina, action, audio, tooltip, midi, code, codeentry }


Layer Structures/Info
Image Layer
Structure
Code: Select all
z        => 2 bytes [int]
subz     => 2 bytes [int]
nooff    => 1 byte [boolean] (Obsolete flag 1: Layered)
id       => 2 bytes [int] (used for identifying the image, used in anim)
o        => 4 bytes [Coord=>(int,int)]
.png byte array [byte[]]

Decoded Files
Image Layers will produce a image_*.png (.png byte array) and a image_*.data (z,subz,nooff,id,o) files.

Tile Layer
Structure
Code: Select all
t    => 1 byte [char]
id   => 1 byte [int]
w    => 2 bytes [int]
.png byte array [byte[]]

Decoded Files
Tile Layers will produce a tile_*.png (.png byte array) and a tile_*.data (t,id,w) files.

Neg Layer
Structure
Code: Select all
cc => 4 bytes [Coord=>(int,int)]
bc => 4 bytes [Coord=>(int,int)]
bs => 4 bytes [Coord=>(int,int)]
sz => 4 bytes [Coord=>(int,int)]
en => 1 byte [int]
for ( i = 0; i -> en)
   epid => 1 byte [int]
   cn   => 2 bytes [int]
   for( j = 0; j -> cn)
      ep[epid][j] => 4 bytes [Coord=>(int,int)]

Decoded Files
Neg Layers produce a single neg_*.data file containing all the information
Notes:
1) one of the coordinates in here is actually the hitbox (changing it won't actually change the hitbox, that's server side info, but this could be useful for someone wanting to write path finding stuff) of the object refer to:
https://github.com/dolda2000/haven-clie ... pView.java
Code: Select all
if(Config.bounddb && ui.modshift) {
   g.chcolor(255, 0, 0, 128);
   synchronized(glob.oc) {
      for(Gob gob : glob.oc) {
         Drawable d = gob.getattr(Drawable.class);
         Resource.Neg neg;
         if(d instanceof ResDrawable) {
            ResDrawable rd = (ResDrawable)d;
            if(rd.spr == null)
               continue;
            if(rd.spr.res == null)
               continue;
            neg = rd.spr.res.layer(Resource.negc);
         } else if(d instanceof Layered) {
            Layered lay = (Layered)d;
            if(lay.base.get() == null)
               continue;
            neg = lay.base.get().layer(Resource.negc);
         } else {
            continue;
         }
         if((neg.bs.x > 0) && (neg.bs.y > 0)) {
            Coord c1 = gob.getc().add(neg.bc);
            Coord c2 = gob.getc().add(neg.bc).add(neg.bs);
            g.frect(m2s(c1).add(oc),
            m2s(new Coord(c2.x, c1.y)).add(oc),
            m2s(c2).add(oc),
            m2s(new Coord(c1.x, c2.y)).add(oc));
         }
      }
   }
   g.chcolor();
}


Anim Layer
Structure
Code: Select all
id  => 2 bytes [int]
d   => 2 bytes [int] (duration of said animation [ms])
len => 2 bytes [int] (length of ids int[] array)
for(i=0; i -> len)
   ids[i] => 2 bytes [int] (id of image to be in the animation [image.id])

Decoded Files
Anim Layers produce a single anim_*.data file containing all the information

Tileset Layer
Structure
Code: Select all
fl       => 1 byte [int]
flnum    => 2 bytes [int]
flavprob => 2 bytes [int]
for(i = 0; i -> flnum){
    fln[i] => String  [String]
    flv[i] => 2 bytes [int]
    flw[i] => 1 byte [int]
}

Decoded Files
Tileset Layers produce a single tileset_*.data file containing all the information

Pagina Layer
Structure
Code: Select all
text => String [String]

Decoded Files
Pagina Layers produce a single pagina_*.data file containing all the information
Notes
Used in Lore text for skills : gfx/hud/skills/*

Action Layer
Structure
Code: Select all
pr   => String [String] (full name from base res )
pver => 2 bytes [int] (version)
name => String [String] (name visible to you on client)
preq => String [String] (prerequisite skill needed for it)
hk   => 2 bytes [char]
len  => 2 bytes [int] (length of ad String[])
for( i = 0; i -> len)
    ad[i] => String [String]

Decoded Files
Action Layers produce a single action_*.data file containing all the information
Notes
1) in Resource.java this is the `AButton' class
2) these can be found in all of the paginae *.res files

Audio Layer
Structure
Code: Select all
raw => byte array [byte[]]

Decoded Files
Audio Layers produce a single audo_*.ogg file which when played through say VLC you'll hear its sound

Tooltip Layer
Structure
Code: Select all
text => String [String] (This is the words that appear on tooltips when you mouse over stuff)

Decoded Files
Tooltip Layers produce a single tooltip_*.data file containing all the information

Midi Layer
Structure
Code: Select all
raw => byte array [byte[]]

Decoded Files
Music Layers produce a single midi_*.midi files
Note
2)I don't know what format these actually are hence midi_*.midi

Code Layer
Structure
Code: Select all
name => String [String] (this is the class name)
data => byte array [byte[]] (this is the actual name.class file)

Decoded Files
Code Layers produce a code_*.data file containing all the information and a code_*.class file containing the actual compile class
Notes
1) If you wish to decompile these, use google to find a decompiler
2) Note that when you try to recompile these,etc you'll need their actual name.java => name.class along with the hnh client since it relies on it. When you go to put these back into decoded format you'll have to rename them in the format `code_i.data' & `code_i.class' where `i' is an integer from [0->oo). code_i.data must contain the actual name of the code_i.class in it.
3) if you need examples of these look under the `ui' folder in res, almost all of them contain code layers
4) from my understanding code only works for making ui widgets

Codeentry
Structure
Code: Select all
while( not reached end of buffer ){
     p[x] => String [String] (key => class type)
     e[x] => String [String] (value => class maker)
}

Decoded Files
Codeentry Layers produce a single codeentry_*.data file containing its data
Notes
1) these have to be present with code layers
2) p[x] refers to the type of the class (ie: wdg for Widget.java)
2) e[x] refers to the classes Maker which makes said class and returns it as type p[x]




Extended Layer Help
Tooltip Layer
Tooltips are neat in a way since they are parsed/rendered such that they can have added benefits.
If you were to put a newline in the middle of your tooltip, it would see this and render accordingly.
Example: Image
which came from :
Code: Select all
#TOOLTIP LAYER FOR RES dout//linenshirt.res
#String tooltip
Linen Shirt\nBase Armor Class: 0\n

(yeah it's a little ugly example since the the client appends on `, quailty 10' at the end, but it's meant to show the concept off)
refer to https://github.com/dolda2000/haven-clie ... hText.java for parsing information (this might work for action layer too for the display of an icon, but i haven't tested nor looked)

Anim Layer
The whole reason I even made this Utility was originally due to people in another thread wondering how to add more/new animation to hnh.
So i took the time to figure this out or how i think it should work. I'll be using hnh code to explain this.

So lets start off in the Resource.java within the Anim class (just read the comments to understand it, they'll walk you right through):
https://github.com/dolda2000/haven-clie ... ource.java
Code: Select all
/* Animation class */
public class Anim extends Layer {
   /* Array of ints which refer to the ids of the Image Layer's `id' [int] variable
      this is used to identify which .png's should be part of this single animation */
   private int[] ids;
   /* The id [int] of this single Animation */
   public int id;
   /* The duration of this single animation [i suspect it's in milliseconds]
      and by duration i mean how long a single image in the animation stays up
      for */
   public int d;
   /* a 2D array of type Image Layer which will contain the Image layers that are meant
      to be part of this animation such that Image.id is contained in ids int array */
   public Image[][] f;

   /* Animation constructor */
   public Anim(byte[] buf) {
      /* read in the id of this animation [2 bytes] */
      id = Utils.int16d(buf, 0);
      /* read in the duration of this animation [2 bytes] */
      d = Utils.uint16d(buf, 2);
      /* init the ids array with [2 bytes] len */
      ids = new int[Utils.uint16d(buf, 4)];
      /* This rule basically means that all the information left in the buffer
         refers to the ids that'll be placed into the ids int[] */
      if(buf.length - 6 != ids.length * 2)
         throw(new LoadException("Invalid anim descriptor in " + name, Resource.this));
      /* for each id in the ids int[] */
      for(int i = 0; i < ids.length; i++)
         /* read in the id of the Image Layer [2 bytes each]*/
         ids[i] = Utils.int16d(buf, 6 + (i * 2));
   }

   /* Init the animation class, this is called after all Layers of said Resource are constructed*/
   public void init() {
      /* Make the 2D Image Layer array have 'x' rows such that x = ids.length */
      f = new Image[ids.length][];
      /* this single array is merely used later on for coverting the linked list to a array */
      Image[] typeinfo = new Image[0];
      /* for each id in ids */
      for(int i = 0; i < ids.length; i++) {
         /* create a linkedlist to contain all images */
         LinkedList<Image> buf = new LinkedList<Image>();
         /* search through all our Image layers in the resource file as of right now*/
         for(Image img : layers(Image.class)) {
            /* if this img.id is the same as ids[i] */
            if(img.id == ids[i])
               /* add to our linkedlist */
               buf.add(img);
         }
         /* convert the linkedlist to a 1D array and sort at f[i] */
         f[i] = buf.toArray(typeinfo);
      }
   }
}/* End Animation Class */


Now by now you should understand how the animation class works or atleast get the point of it, but where else in the code is animations actually mentioned?
The AnimSprite.java -> https://github.com/dolda2000/haven-clie ... prite.java
note: this is just a snip-it of the parts that were important (i think), once again read the comments
Code: Select all
private AnimSprite(Owner owner, Resource res, Message sdt) {
   super(owner, res, sdt);
   boolean[] flags = decflags(sdt); /* Ignore this */
   Collection<Part> stp = new LinkedList<Part>(); /* Ignore this */
   for(Resource.Image img : res.layers(Resource.imgc)) { /* Ignore this for */
      if((img.id < 0) || ((img.id < flags.length) && flags[img.id]))
         stp.add(new ImagePart(img));
   }
   frames = null;
   /* For every Animation Layer contained in this Res's layers */
   for(Resource.Anim anim : res.layers(Resource.animc)) {
      /* anim.id either has to be < 0 OR (anim.id < flags.length AND flags[anim.id] is true)
         for us we'll want to keep anim.id < 0 to avoid the second half of that OR statement*/
      if((anim.id < 0) || ((anim.id < flags.length) && flags[anim.id])) {
         if(frames == null) {
            /* init frames to be the length of the first D in our 2D f Image Layer array in anim */
            frames = new Frame[anim.f.length];
      } else {/* skip this */
         if(anim.f.length != frames.length)
            throw(new ResourceException("Attempting to combine animations of different lengths", res));
      }
      
      /* for each frame */
      for(int i = 0; i < frames.length; i++) {
         Frame f;
         if(frames[i] == null) {
            f = frames[i] = new Frame();
            /* Frames duration is == anim duration */
            f.dur = anim.d;
            for(Part p : stp) /* skip this for us */
               f.parts.add(p);
            f.id = (res.name + ":" + res.ver + ":" + i).intern();
         }
         f = frames[i];
         /* for each Image Layer in the 2D Image Layer Array in anim */
         for(int o = 0; o < anim.f[i].length; o++)
            /* add it to the frame */
            f.parts.add(new ImagePart(anim.f[i][o]));
         }
      }
   }
   
   if(frames == null) { /* ignore this */
      frames = new Frame[1];
      Frame f = frames[0] = new Frame();
      f.dur = 10000;
      f.parts = stp;
      f.id = (res.name + ":" + res.ver).intern();
   }
   
   fno = 0;
   te = 0;
   /* set current frame to the first frame */
   curf = frames[0].parts;
}
   
/* this basically changes the image in the animation based on the
   duration */
public boolean tick(int dt) {
   boolean rv = false;
   te += dt;
   while(te > frames[fno].dur) {
      te -= frames[fno].dur;
      if(++fno >= frames.length) {
         fno = 0;
         rv = true;
      }
   }
   curf = frames[fno].parts;
   return(rv);
}


So from this I concluded that:
1) Each Animation Layer refers to a single animation (ie: bees flying is one animation layer)
2) d controls the duration of our animation from one image to the next
3) Keep the anim.id < 0 (-1) to avoid the parts i put /* ignore this */

Could i be wrong? yeah, but i think my ideas are correct here.

Lets use the beehive terobjs as an example (bhive.res)
First of all we know that beehives have 1 animation : the flying of the bees
After decoding bhive.res, there's 1 anim layer found. So far this goes right along with my idea of 1 animation = 1 anim layer
Code: Select all
#ANIM LAYER FOR RES dout//bhive.res
#int16 id [keep -1]
-1
#uint16 d [duration of animation]
100
#uint16 ids [length]
8
#uint16 ids[0]
128
#uint16 ids[1]
129
#uint16 ids[2]
130
#uint16 ids[3]
131
#uint16 ids[4]
132
#uint16 ids[5]
133
#uint16 ids[6]
134
#uint16 ids[7]
135


So our ids is of length 8, and the following were the images part of this single animation { 128,...,135 }.
When you go to examine the .data files for each Image layer in bhive you'll find that the set of Image Layer with id { 128,...,135 } refer to images of the flying bees
So this one goes right along with my thoughts on how animation works.

Now have i tested new animation to confirm my thoughts? no, that's where someone that wishes to further his hnh experience by making these modifications and dedicate time to figuring this stuff out even further comes in. So if there's someone that wants to add animation, use my idea or form your own and go at it now that you have access to decoded anim layer and you can re-encode them afterwards to test.

Initial tests of my idea on Animation:
http://odditown.taggedup.com/boshaw/Lay ... im.res.rar
Status: Success, it's goofy but it shows that you can add animation to any resource. I have it packaged so you can see my decoded files along with a encode claim.res file for use to see it. So if needed use this as a play test, change values in the Image Layer .data files see what they do and figure out how to make this stuff work better and share with us.


I'll add more extended layer help once i take the time to figure out more stuff.


Disclaimer
Code: Select all
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPY RIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR
THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Last edited by boshaw on Thu Jan 29, 2015 12:24 am, edited 21 times in total.
User avatar
boshaw
 
Posts: 1538
Joined: Tue Jun 01, 2010 10:22 pm

Re: Resource Layer Utility

Postby sabinati » Mon Nov 07, 2011 3:16 am

you truly are a wizard, sir
User avatar
sabinati
 
Posts: 15497
Joined: Mon Jul 13, 2009 4:25 am
Location: View active topics

Re: Resource Layer Utility

Postby EyeOfRa » Mon Nov 07, 2011 3:19 am

I have no idea what any of this means... Will probably download anyway.
God forgives, Melsonia does not
Image -RJT on Melsonians o7
User avatar
EyeOfRa
 
Posts: 453
Joined: Thu Apr 07, 2011 2:14 am

Re: Resource Layer Utility

Postby boshaw » Mon Nov 07, 2011 5:44 am

Update:
Current Bugs:
Action Layer encoding, see first post at top for more info.
Edit: fixed Action Layer


Initial tests of my idea on Animation:
http://odditown.taggedup.com/boshaw/Lay ... im.res.rar
Status: Success, it's goofy but it shows that you can add animation to any resource. I have it packaged so you can see my decoded files along with a encode claim.res file for use to see it. So if needed use this as a play test, change values in the Image Layer .data files see what they do and figure out how to make this stuff work better and share with us.
User avatar
boshaw
 
Posts: 1538
Joined: Tue Jun 01, 2010 10:22 pm

Re: Resource Layer Utility

Postby tempwad » Mon Nov 07, 2011 6:34 am

i have a question about tooltips: why they have no cyrillic fonts?
tried to send russian letters, but got a string of squares.
can you research this shit?
trapped? masturbate.
User avatar
tempwad
 
Posts: 383
Joined: Thu Feb 10, 2011 1:03 pm

Re: Resource Layer Utility

Postby Nummy » Mon Nov 07, 2011 7:46 am

Much thanks, makes a life about modifying client much much easier :)
User avatar
Nummy
 
Posts: 489
Joined: Sat Mar 19, 2011 10:01 am

Re: Resource Layer Utility

Postby boshaw » Mon Nov 07, 2011 3:45 pm

tempwad wrote:i have a question about tooltips: why they have no cyrillic fonts?
tried to send russian letters, but got a string of squares.
can you research this shit?


paste on here what text you're trying to type, it's something i did when encoding/ loading in decoded files.basically I need to add utf-8 support which I just did but now the characters show up as blank spaces, so later today ill look into it
User avatar
boshaw
 
Posts: 1538
Joined: Tue Jun 01, 2010 10:22 pm

Re: Resource Layer Utility

Postby Daji » Mon Nov 07, 2011 6:03 pm

Roooooh, life is rainbow of eight colors
User avatar
Daji
 
Posts: 321
Joined: Tue Jan 18, 2011 9:32 pm
Location: I'm a wizard!

Re: Resource Layer Utility

Postby SacreDoom » Mon Nov 07, 2011 6:14 pm

This is wonderful Boshaw.

Might even make that rabbit-burner now. :)
User avatar
SacreDoom
 
Posts: 2590
Joined: Tue Apr 06, 2010 3:14 pm

Re: Resource Layer Utility

Postby boshaw » Mon Nov 07, 2011 9:28 pm

Update:
Bug: UTF-8 problems when encoding/decoding
Bug is now fixed and LayerUtil should support UTF-8 deocded files and encode UTF-8 strings correctly now.

Image

Bug: tilesets header was encoded incorrectly by one byte
Bug is fixed now, before it was encoding tilesets as `tiiesets\0', it's fixed now at `tilesets\0' like it should be



github and compiled version should both be updated now.

If you file anymore errors tell me.

Also i know some text editors will put in random bytes at the very beginning of a file when you enable UTF-8 encoding (notepad++ does this), those random bytes need to be removed otherwise it'll error out the LyaerUtil.java this can be done with a hex editor, eventually i'll find a way around this.

SacreDoom wrote:Might even make that rabbit-burner now. :)

You better, I've been waiting a long time for it now.
User avatar
boshaw
 
Posts: 1538
Joined: Tue Jun 01, 2010 10:22 pm

Next

Return to The Wizards' Tower

Who is online

Users browsing this forum: No registered users and 2 guests