Auditor Help: Ecora CSV Architecture


CSV (comma separated values) files are one type of output generated by the Ecora software. This document describes the structure (schema) of information contained in the *.csv files generated by Ecora software and explains the way in which the CSV files are inter-related.

The CSV files contain raw configuration data, which can be loaded into a database for granular analysis. The content of Ecora's software reports can be viewed as a set of containers, where each container is composed of a set of attributes or other containers. For example, a domain container is composed of a simple attribute indicating the name of the domain and a container holding a list of servers (and possibly other information). A server container is composed of many simple attributes and container objects. Ecora's software generates a *.csv data file for each container type used in the model of our products. For example, the raw configuration data for the hardware container is contained in the hardware.csv file.

Consider a system analyzed by Ecora software. Conceptually, the system has a hierarchical structure whose members are container types. Each container is composed of the following:

  • simple attributes (integer or string), or
  • other containers.

There is one top-level container, which is not contained in any other object, and this object is the root (top) of the hierarchical structure. In the Ecora software, the root is the WindowsNetwork container (WindowsNetwork.csv).

For each kind of container, CONTAINER, Ecora software produces a CONTAINER.csv file describing all objects of the CONTAINER type in the system. These *.csv files can be mapped to relational tables for complex queries across various container csv files (tables).

The *.CSV Files

The rules used to build these *.csv files are the following:

1. Static Information: The schema of the *.csv files can be obtained in two ways:

1.1. It is described by the two files: csv.sch and

1.2. It can be inferred from the model.xml file.

2. Dynamic Information: The content of the .csv files is determined at run time (when the Ecora software is run). The content of the *.csv files depends on the data collected by the software during a particular run and may differ for different runs. But the schema of the data in the *.csv files is fixed and defined as outlined at (1).

3. The name for the *.csv file of a container "C" is formed by appending the .csv extension to the container name, such as C.csv.

4. There are two special *.csv files for the simple (non-composite) types: "int.csv" and "wstring.csv"

5. Consider a container "Container" that has a member "attr" of type "T."

5.1. If "T" is a simple type (integer or string), "attr" is placed in the Container.csv file/table (on the "attr" column of the table).

5.2. If "T" is a container, "attr" gets dumped into T's own table, T.csv.

5.3. If "T" is a set of elements of "Base" type, each element of the set gets dumped into Base's table, Base.csv. There will be one entry in Base for each element of the set.

NOTE:  All simple members of a container are placed into that container's csv file, into columns described by their name. All members are placed into other containers' .csv files.

6. The schema is described in csv.sch and files.

6.1 csv.sch

For every container type T, csv.sch contains a line for the corresponding T.csv table. This line has the following format:

:SnapshotCreationTime, Id, ContainerType, ContainerId, Role [,ColumnList]

where ColumnList is a (possibly empty) list of columns that correspond to the simple attributes of T. Each attribute is identified by its name in this list and it may contain an optional description in a parenthesis after the attribute name.

To describe the meaning of the first five attributes (present on all lines of any .csv file), consider an object X of type T that is placed into T.csv. Assume we found X while we were looking at a member "X_name" inside a container object C of type CONT. Then the meaning for the first five columns for X' entry in T.csv is:

  • SnapshotCreationTime: the time when the entire data was collected.
  • Id: X's (unique) identifier inside its own table.

Note: the "Id" column is missing from the special two tables int.csv and wstring.csv).

  • ContainerType: the name of C's type: CONT.
  • ContainerId: C's Id (unique identifier) inside CONT's table (which is given by ContainerType).
  • Role: X's name (role) inside CONT. (That is "X_name", in our example).

6.2 The file.

Items 5.2 and 5.3 mention cases when a member X of some other object C, is placed to its own table. Assume that X's type is T and that C's type is CONT, then X gets placed into T.csv, while C gets placed into CONT.csv. Then, while examining X's entry in T.csv, how can we recover all the information about its container? The answer is: by using links from T's table back to CONT's table. In fact, the information needed for these "back links" is given by ContainerType, ContainerId and Role fields described at item 6.1. The schema information for these back links is described in contains a line for each table Container.csv (except for the tables corresponding to the two simple types: int.csv and wstring.csv), having the following format:

Container: [,BackLinkList]

   BackLinkList is a (possible empty) list describing the members of Container that are placed in a file (table) other than Container.csv file (table.) Consider a Container that contains a member "attr" of type T. If the attr were placed in T's table (rather than in Container's table), then the Container's entry in the would contain:

  • T:attr
    This indicates Container contains a member that is placed into the T.csv table and the role is "attr". So, if attr were the only member of Container placed elsewhere, then Container's entry in the would be:
    • Container:,T:attr
  • An optional description may be added in parenthesis after the role name:
    • Container:,T:attr(some description)
  • If "attr" would represent a set of T (rather than a T) inside Container then the Container's entry in the would be:
    • Container:,[T]:attr(some description)

The square brackets around T show that the object with the role "attr" represents a set of T.

Example 1

The following is an example of a line from the file for Ecora software for Windows. Consider the Hardware container described in Example 1. The information in the csv.sch file does not specify other containers that are part of the Hardware container (for example DiskDrives, Modems and NetworkAdapter).

The file contains the information about this relationship.

Hardware:, [wstring]:HardwareSystemBiosVersion(The Version the motherboard BIOS.), [wstring]:HardwareVideoBiosVersion(The version of the video card BIOS.),
DiskDrives:DiskDrives(Disk drive object. ),
Modems:Modems(Container for modem objects),
NetworkAdapter:NetworkAdapter(Network adapter object.)

Note: please see the associated csv.sch file for a list of attributes that compose the Hardware container and subcontainers.

The data for HardwareSystemBiosVersion will be found in the wstring.csv file the role column will indicate HardwareSystemBiosVersion.

The Hardware.csv may contain:


SnapshotCreationLocalTime, ObjectId, ContainerName, ContainerId, Role, DomainServer, FileIndex, HardwareCPUType, HardwareCPUSpeed, HardwareCPUCount, HardwareSystemBiosDate, HardwareVideoBiosDate, HardwarePhysicalMemory, Hardware_model
May-01-2002_17-32-47,1,Server,1,Hardware,BSP/PHOENIX,,x86 Family 6 Model 6 Stepping 2, 1194, 2, 11/02/01, 03/20/00, 1023,AT/AT COMPATIBLE

The wstring.csv may contain:

May-01-2002_17-32-47,Hardware,1,HardwareSystemBiosVersion,Phoenix ServerBIOS 2 Release 6.0

Example 2 (static)

/// example2.eml:
root: Organization;
class Organization
Person TheBoss;
vector<Client> Clients;
vector<Team> ChessTeams;

class Client
Person Contact;
vector<Team> ClientsChessTeams;

class Team
string TeamName;
string CoachName;
vector<string> GameDates;
Person Captain;
vector<Person> Player;

class Person{
string LastName;
string FirstName;
/// csv.sch:
Organization:SnapshotCreationTime, Id, ContainerType, ContainerId, Role
Team:SnapshotCreationTime, Id, ContainerType, ContainerId, Role, TeamName(This is the team name), CoachName(Team's Coach Name)
Person:SnapshotCreationTime, Id, ContainerType, ContainerId, Role, LastName, FirstName
Client:SnapshotCreationTime, Id, ContainerType, ContainerId, Role

Organization:, Person:TheBoss, [Client]:Clients, [Team]:ChessTeams(Recreational chess teams.)
Team:, [wstring]:GameDates(Game dates), Person:Captain, [Person]:Player
Client:, Person:Contact, [Team]:ClientsChessTeams

So, if we try to recover data about a Team object, Y, from the
.csv files and we are interested in the value of its Captain, we
should look into the Person.csv table for an element whose ContainerType
is "Team" and whose ContainerId is the same as Y's ID inside the
Team.csv table *AND* whose Role is "Captain."

Example 3 (dynamic)

With the structure from example 2, assume that we have several chess teams:

---- chess team 1 ---
TeamName: PagesTeam
CoachName: Carl
GameDates: ["jan 1","feb 1","june 1"];
Captain: {Page, Keaton}
Players: [
{Gilde, Bend},
{Carl, Senter},
{Joe, Schmoe}
---- chess team 1 ---
TeamName: KevinsTeam
CoachName: Dave
GameDates: ["jan 1","feb 1","Aug 1"];
Captain: {Dave, Labins}
Players: [
{Kevin, McGuire},
{Dan, Pasil},
{Virginia, Karus}

The Team.csv may contain:

"Monday, September 24, 2003 15:51:02",4,Company,1,ChessTeams,PagesTeam,Carl
"Monday, September 24, 2003 15:51:02",5,Company,1,ChessTeams,KevinsTeam,Dave
"Monday, September 24, 2003 15:51:02",6,Company,1,ChessTeams,ChrisTeam,Weston
"Monday, September 24, 2003 15:51:02",7,Company,3,ClientsChessTeams,GatesTeam,Balmer

The Person.csv may contain:

"Monday, September 24, 2003 15:51:02",1, Organization,1,TheBoss,Virginia,Karus
"Monday, September 24, 2003 15:51:02",21,Team,4,Captain,Page,Keaton
"Monday, September 24, 2003 15:51:02",22,Team,4,Players,Gilde,Bend
"Monday, September 24, 2003 15:51:02",23,Team,4,Players,Carl,Senter
"Monday, September 24, 2003 15:51:02",24,Team,4,Players,Joe,Schmoe
"Monday, September 24, 2003 15:51:02",25,Team,5,Captain,Dave,Labins
"Monday, September 24, 2003 15:51:02",26,Team,5,Players,Kevin,McGuigg
"Monday, September 24, 2003 15:51:02",27,Team,5,Players,Dan,Pasil
"Monday, September 24, 2003 15:51:02",28,Team,5,Players,Virginia,Karus

The wstring.csv may contain:

"Monday, September 24, 2003 15:51:02",Team,4,GameDates,Organization,jan 1
"Monday, September 24, 2003 15:51:02",Team,4,GameDates,Organization,feb 1
"Monday, September 24, 2003 15:51:02",Team,4,GameDates,Organization,june 1
"Monday, September 24, 2003 15:51:02",Team,5,GameDates,Organization,jan 1
"Monday, September 24, 2003 15:51:02",Team,5,GameDates,Organization,feb 1
"Monday, September 24, 2003 15:51:02",Team,5,GameDates,Organization,aug 1