Noesis - 3D model viewer and extraction tool

Noesis is a program for viewing, extracting, and converting digital assets from video games, and was created by a guy named Rich Whitehouse. You can visit his personal website at richwhitehouse.com, or follow him on Twitter @DickWhitehouse. Unlike the FFXI Model Viewer and Altana Viewer, Noesis is a universal model viewer designed to work with (almost) any video game. Thankfully, FFXI happens to be one of the games it works with.

You can get Noesis here.

Advantages:

• Can view any .DAT file, including zones, not just character models.
• Can export .DAT file models directly into several standard file formats, including .obj and .fbx formats.
• Unless specifically disabled by the user (and this is a huge selling point), character models retain skeletons and animations upon export.

Disadvantages:

• Because Noesis was designed as a universal model viewer for all games, and not a model viewer specifically for any one particular game, the process for loading models involves just using a standard file explorer window for loading files, which makes it a pain to find things if you don't already know exactly where to look. It's recommended that you use either the FFXI Model Viewer or Altana Viewer to find the file locations of the character models you want to load before you go browsing through the FFXI game folders via Noesis. Good luck figuring out which files to load for zones, though.
• The method for accessing model data for player characters in particular is extremely difficult to use. The process requires creating a custom text file, which is then opened in Noesis. Details on how to create this special text file are provided later in this post.

Download:
Noesis can be downloaded for free from the Resources section on Rich Whitehouse's personal website, richwhitehouse.com. The direct link to Noesis version 4.31 is listed below:

Direct Download: Noesis — version 4.31
(22.28 MB) - (uploaded: 7/14/2018)

It's possible there may have been new versions of Noesis released since I wrote this blog entry. If you prefer to get the most recent version, Rich Whitehouse keeps a page in the Projects section on his personal website updated with the latest version of Noesis, which you can access at the following link:

Noesis — Official Project Page

Alternatively, you can also update your version of Noesis after download by going to "Tools" → "Check for Updates" in the main toolbar in Noesis. Using the latest version is recommended.

XeNTaX Game Research Forum:
Additionally, there is a game modding forum called XeNTaX that has an old topic which is specifically about using Noesis with FFXI, and has some useful information (Rich Whitehouse is the moderator with the screen name MrAdults). You can read the topic here:

XeNTaX: Game Research Forum » Topic: "Final Fantasy XI .DAT (PC)"

Code Snippets:
Rich Whitehouse has provided some of his code on his website.

richwhitehouse.com » Resources » Snippets

Viewing Player Character Models:
Viewing, displaying and exporting player character models is extremely tricky in Noesis. To do it, you must create a special scene file using one of two different custom file types created by Rich Whitehouse. The reason for there being two different files types is because everyone who has ever played FFXI knows that the game actually has two sets of character models for player characters. There is the standard, low-resolution player character models that are actually used in-game, and then there's a completely different set of special high-resolution character models which appear only during character creation and then are never seen again. Noesis gives us the ability to view and export both types of character models. The Noesis scenes directory located at "C:\...\noesisv42\scenes" has two example files: one called "sample_elf01ff11.noesis" which is used for the high-resolution character models from the character creation screen, and another called "sample_hume01.ff11datset" which is used for the low-resolution character models from in-game. Both of these file formats can be edited with Notepad or any other text editor.

Hi-Rez Player Character Models (From Character Creation):
In order to view the high-resolution character models which appear only during character creation, you must use this special format of text file called ".noesis", which is one of the custom file extensions Rich Whitehouse created. There is example file called "sample_elf01ff11.noesis" located in your Noesis scenes directory at "C:\...\noesisv42\scenes". When opened with Noesis, this file will display the high resolution character model of the female Elvaan as she appears during character creation. Please note, however, that when Rich Whitehouse wrote this particular scene file, he specified only a partial file path, which results in the scene file only working if you place it inside your root FFXI directory, which should be located at "C:\Program Files (x86)\PlayOnline\SquareEnix\FINAL FANTASY XI," unless of course you specified a custom filepath when you installed FFXI. If you want to save this Noesis scene file anywhere on your computer and still have it work, you will have to specify the full file path in the model line, instead of only the partial path  If only the partial path is specified and not the full file path, and you try to open this Noesis scene file while it is located in any other folder besides the FFXI root directory, you will get an error message. Specify the full file path to avoid the error message.

The following is a post that Rich Whitehouse made on the XeNTaX game research forum explaining how to use the file format:

Originally Posted by MrAdults (Rich Whitehouse)
(Click here to view original post.)

---

To piece the SQLE models together, you can put a .noesis file in your FFXI directory root (above the ROM directories), like the sample_elf01ff11.noesis file included in the latest build. It looks like this:

sample_elf01ff11.noesis:

NOESIS_SCENE_FILE
version 1
physicslib ""
defaultAxis "0"

object
{
name "body"
model "ROM/63/1.dat"
loadOptions "-ff11sqleanim ../64/40.dat"
}
object
{
name "head"
model "ROM/63/5.dat"
loadOptions "-ff11sqleanim ../64/46.dat"
;this uses the relative positions of the neck joints on each skeleton to place the head
offsetWithBones "bone0001" "body" "bone0004"
;combine both objects into a single model
mergeTo "body"
}

Modifying for other body/head/animation combinations should be pretty obvious. The bones for the head snapping may change, so figure out which ones you should be using by looking at the models in the Noesis data viewer.

Low-Rez Player Character Models (From In-Game):
In order to view the low-resolution character models which are used within the actual game itself, you must use this special format of text file called ".ff11datset" which is a special custom file format that Rich Whitehouse created specifically for use with FFXI in Noesis. You can use this to view and extract the standard in-game player character models with a specific set of armor. There is an example file called "sample_hume01.ff11datset" in the Noesis scenes directory, which shows how to set up the file for a male Hume.

Unlike the Noesis scene files, the FFXI Dat Set file does not require the full file path to be specified in order to work outside of the FFXI root directory.

Like the other scene file, this file also can be edited with Notepad or any other text editor. You can use the FFXI Model Viewer or Altana Viewer to figure out which DAT files to specify in the text file.

sample_hume01.ff11datset:

NOESIS_FF11_DAT_SET
;^ must be the first line of the file

;search for dats using a path retrieved from a registry key
setPathKey "HKEY_LOCAL_MACHINE" "SOFTWARE\PlayOnlineUS\InstallFolder" "0001"

;search for dats on a path relative to this file
;setPathRel "./"

;search for dats on an absolute path
;setPathAbs "c:/whatever/ff11/"

dat "__skeleton" "ROM/27/82.dat"
dat "__animation" "ROM/27/82.dat"
dat "face" "ROM/27/87.dat"
dat "head" "ROM/27/103.dat"
dat "body" "ROM/28/7.dat"
dat "hands" "ROM/28/52.dat"
dat "waist" "ROM/28/84.dat"
dat "legs" "ROM/28/116.dat"
dat "weapon" "ROM/29/20.dat"

Loading and Exporting Zones:
There are two ways to load zone models in Noesis. The first is to simply go to "File" → "Open File" on the main toolbar, and then browse the folders on your PC to find the DAT file you want to load. The second method is to create a special Noesis scene file, place that scene file in your root FFXI directory, and then open that scene file instead of directly opening the DAT file. Using this second method allows you to specify several advanced load options which you cannot specify using the first method.

Advanced Load/Export Options:
The following is a list of special options specifically for FFXI which can be used when loading a model in Noesis:

• -ff11blendhack <arg>: Software render to determine alpha coverage, <arg>=minimum fraction.
• This uses the Noesis software renderer to render all FF11 map geometry triangles in UV space, and determine how many of each mesh's triangles are only touching zero-alpha pixels in the texture. -ff11blendhack 0.99 is typically a good value to use. This completely demented solution is not foolproof, but catches the vast majority of broken blending cases.
• -ff11renderunref: Render unreferenced map geometry at identity.
• Gets model data about sky/moon/water/etc., and spits models out at identity. You have to figure out how to pull them apart and place them yourself if you want to do something with them.
• -ff11keepnames: Keep map object names intact.
• Preserves the names of mesh geometry pieces in the 3D model. Use in conjunction with the above option to avoid having a big terrible inseparable triangle mess.
• -ff11optimizegeo: Remove degenerates and redundant vertices.
• Collapse redundancies and remove strip degenerates when exporting geometry. Unlike the other options above, which are invoked when loading a model into Noesis, this command is invoked when exporting a model from Noesis.
• -ff11sqleanim <arg>: Load SQLE animation <arg> as relative path.
• Option for SQLE animations. Used only for loading the hi-rez character models from the character creation screen.

With the exception of the "-ff11optimizegeo" command, the above options do not do anything if you try to input them into the Advanced Options box on export (explained below). These advanced options can only be invoked when loading zone models, otherwise the model will be exported without these options enabled, even if you try to input them in the export window. In order to use Advanced Options, you have to create a special Noesis scene file, like the one used for the hi-rez character models above, and use the Noesis extension as the file type (i.e. "filename.noesis"). The following is an example scene file for importing zones into Noesis using a scene file:

FFXI_Zone_BastokMarkets.noesis:

NOESIS_SCENE_FILE
version 1
physicslib ""
defaultAxis "0"

object
{
name "Environment"
model "C:/Program Files (x86)/PlayOnline/SquareEnix/FINAL FANTASY XI/ROM/1/35.dat"
loadOptions "-ff11blendhack 0.99 -ff11renderunref -ff11keepnames"
}

Exporting Zones:
In my personal experience, I have found that it's usually best to use the FBX file format for saving the model data (that's the Main output type), and TGA as the file format for textures. Saving the textures as BMP files will destroy their alpha channel, which is often needed for transparency on many objects, while saving the textures as PNG files preserves the alpha channel but causes the textures to not map correctly to the corresponding geometry for some mysterious reason. Saving the textures as TGA files avoids both of these problems.

Another common issue is that for some reason, when you export FFXI zones from Noesis, by default the zones get flipped upside down in most 3D modeling programs. This problem can be fixed by specifying a rotate value in Noesis on export. To specify a rotate value, type the command "-rotate" into the "Advanced options" box in the "Export media" window in Noesis, followed by three numbers (separated by spaces) to indicate the number of degrees to rotate along the XYZ axes. The pattern is "-rotate X Y Z." So if we wanted to rotate 180 degrees on the X axis, 0 degrees on the Y axis, and 270 degrees on the Z axis, we would type "-rotate 180 0 270," like so:

-rotate 180 0 270

You can enter as many commands as you like into the "Advanced export" options box. For our purposes here, the following is a good set of advanced options to use when exporting a model:

-rotate 180 0 0 -scale 100 -fbxscalehack -fbxsmoothgroups -fbxnewexport -ff11optimizegeo

That about wraps it up. If you have any questions, feel free to leave a comment below.