Yet Another Everquest® Log file Parser – YAELP

Overview

YAELP is an EverQuest® log file parser designed to create and maintain one or more guild rosters. There are some functions which deal with attributes of your personal characters, but these are limited to skill attributes. If you need more detail and statistics on your own characters, you should look for one of the other parsers out there.

A globally universal and compact save file format allows you to share character roster information with other players through email. Roster information for 300+ characters takes less than 10k of save file size. The same roster, stripped of all extra information and properties and exported in CSV text format takes up more than twice the space.

You will need a recent version of the Java™ virtual machine to run this program. If you do not plan to develop and just want to run the program choose a download from the JRE column for your platform. Some operating systems already come with Java™ preinstalled. Try running java -version from a command line. If it does not report an error and the version printed is 1.3.x or above, you should be fine and won't need to download it.

Hosted by SourceForgeDevelopment is currently done as a part of the fraglets project at sourceforge. You can download the latest release from the files section. Open bugs are tracked here, please submit any new bugs you might happen to find. YAELP is distributed under the GNU Public License which allows you to copy and modify it as long as you keep the license and copyright intact and put the results under the same license.

Installation

Unpack the yaelp-X.XX.zip archive to an empty directory of your choice. There is no installer and no configuration to be done other than unpacking it. Just remember where you put it.

Starting the program

This is something I have not cared for much yet. Procedures vary depending on operation system and configuration. If your platform supports it, you may start the program by double-clicking the yaelp-X.XX.jar file. This is an executable jar file which contains the main program code.

If that doesn't work for you, start a command line interface and change to the directory where you unpacked the archive. Type java -jar yaelp-X.XX.jar to start the program. You may have to provide the full path to the java executable if it is not in your current execution path.

After program initialization a window looking similar like this should open:
Starting YAELP

The File menu.

Figure 1: The File menu This is where you will almost always start working. If you have previouosly used YAELP and saved some of the results, you might want to Load ... these now. This will restore all entries which were visible in the character roster at the time they were saved. Loading a new roster will clear all current entries. You may be asked for confirmation if the program detects that unsaved changes exist.

Status line with all entries hidden The current filter settings remain active when loading new data. Filter settings are not stored in the save file. So if you don't see what you expected, have a look at the status line at the bottom of the window. If there are many characters hidden just clear the filter settings.

Now you may want Parse logs ... to update the roster with new information from the latest log files. YAELP can parse the plain text log files (eqlog_*.txt) as well as gzip compressed log files (eqlog_*.txt.gz). Information in the roster is updated according to the timestamp mechanism. This allows you to parse a bunch of intermixed log files which may be in any order.

Since the time recorded in the log files is the local time according to your time zone, you should not parse log files you got from a different time zone. Dont exchange raw log files, better exchange the saved roster information.

To keep things accurate, make sure your system clock is set correctly. This can be automatically done using a network time client like NetTime. Note that you may have to watch out that this does not constantly open your dialup connection. See the support forum for NetTime if it causes you trouble.

Figure 2: Parse log files The file chooser for parsing tries to find your Everquest® install directory when opened the fist time. Since the average Everquest® directory contains well over a thousand files, the file filter is initialized such that only log files should be visible. If you renamed your log files so that they don't show up here, change the Files of type: setting to All Files in the file chooser.

Scanning this directory takes some time since it contains so many files. Don't panic if no files show up for the first few seconds. Specifically with Java versions 1.4 there is no visual indication that it is still busy searching files; it is working in the background.

Save ... saves the currently visible portion of the roster into a YAELP format save file. Note that this means that all characters excluded from the table are not included in the information saved. So be sure to disable all filters if you want to save all information loaded.

Import ... loads some previously saved YAELP roster and adds the information to the current roster. It uses the same timestamp mechanism as the log file parser. However, timestamps in YAELP save files are timezone independent so that you can exchange these file with other people. Loading automatically detects if the file is compressed, so you can load exported XML as well.

Export table ... stores the current table contents as a comma separated text file. This may be used later to import into a spreadsheet or further manipulate the information with other software. This format contains much less information so you cannot load it again.

Export XML ... saves all roster information in an uncompressed XML file. This is the same as using Save ..., but without the compression step. Since this is the XML input which will be passed to the XSL transformation when creating HTML this option is useful when creating new style sheets to examine the XML for things to match on.

Export HTML ... transforms the current roster list with column and sorting settings into a HTML file using the selected XSL style. Depending on the style used, this creates a complete HTML page for use in your web site, or a fragment to post on forums. Or even a different XML format you may need for another program. In that sense, the menu entry is a bit of a misnomer, but it reflects the most common usage. This is what the test data from the save file section looks when exported with one of the predefined styles:

NameCultureClassLevelTime
Testator Testling Tester 6 2002-07-13

Note that the style used here omits the Guild column, which makes sense when all characters listed are in the same guild anyway.

The Filter menu

The Filter menu New filter menu


The Style menu

The Style menu


Sharing roster information

Importing YAELP save files from other players will be the easiest way to share character roster information. For instance, have your officers send you filtered updates on the guild roster in YAELP file format from their log files and combine them to create a fairly complete guild roster. Since officers are the persons who invite others to the guild, their log files will probably show all the members joining the guild. You should get a Rank property on all guild members who are seen joining the guild, with the timestamp on that property the last time of joining the guild or being promoted.

Timestamps and roster updating

some text explaining timestamps

YAELP Save file format

YAELP save files get a file name extension of .yxr by default. YXR is an abbreviation for YAELP XML Roster. And that is exactly what it is: an XML representation of the roster information, GZIP compressed. If you have the gzip program, you can look at it using gzip -dc myroster.yxr for instance.

Here is a short example of the (uncompressed) XML file contents:
<roster title="Roster">
<heading>
<column id="name">Name</column>
<column id="culture">Culture</column>
<column id="class">Class</column>
<column id="level">Level</column>
<column id="guild">Guild</column>
<column id="zone">Zone</column>
<column id="time">Time</column>
</heading>
<avatar time="1026571151670">
<name>Testator</name>
<culture>Testling</culture>
<class>Tester</class>
<level>6</level>
<guild time="1026571218560">The Testing Testers</guild>
<zone>void</zone>
<time>2002-07-13</time>
<property name="Rank" value="a regular member" time="1026571243120"/>
</avatar>
</roster>

This example contains information for only one character named Testator. For each successive character, the avatar section will repeated. Most tags in the avatar section should be self-explanatory. The time attributes store the timestamp of the last change in milliseconds since midnight, January 1, 1970 UTC. There are three things which get a timestamp: the avatar section as a whole, the guild and each property. These are explained in detail in the timestamps and roster updating section. The extra <time> tag is redundant and only retained for HTML formatting. It contains the avatar time truncated to a date, in ISO format.

The <heading> section is currently only used in styles for HTML output and ignored when loading the file. The column id attributes map to the avatar sub-tags, assigning a table header name to each tag. They are listed in column order, as well as the tags in the avatar sections. Predefined XSL styles make use of that fact so that the transformation becomes much simpler and faster.

CSV file format

Here is an example of a CSV export:
"Name", "Culture", "Class", "Level", "Guild", "Zone", "Time"
"Testator", "Testling", "Tester", "6", "The Testing Testers", "void", "2002-07-13"

Importing CSV to OpenOffice Calc This is the same example as in the save file format section. However, since only information from the table is exported, all timestamp and property information is lost.

If you want to import this into OpenOffice Calc or some other spreadsheet application, use double quote as text delimiter and comma and space as a field delimiter and combine delimiters. This should import the table nice and clean. Note that the screen shot shows the german version so your options may be named differently.


EverQuest is a registered trademark of Sony Computer Entertainment America Inc. © 2002 Sony Online Entertainment Inc.