Wednesday, October 14, 2015

DOT Debug View

In my past post I presented a small hack that used the returned value of the toDOT() method of a variable in the debug view (in Eclipse) to generate a graph in the eclipsegraphviz  view. Today I am presenting the new, improved, version of this.

No more hacking into the Eclipsegraphviz view

After discussing my idea with Rafael Chaves (@abstratt), the author of Eclipsegraphviz, he was kind enough to provide a nicer method to access the view provided by his plugin. Thanks a lot Rafael!

The DOT Debug View plugin

My initial hack didn't play nicely with with the thread management principles of both the Elcipse UI and the JDT. Hopefully this has been corrected, and thus using the new plugin should not had adverse effects on the debugger or the UI.

Providing an Eclipse Update Site


When developing plugins I always had the issue of distribution and subsequent updates. Eclipse provides a very solid update framework through via its p2 component. However, if you don't own a domain or hosting space, making your updatesite available might be tricky. 

Luckily, I fumbled upon bintray while looking for alternatives. Based on the script provided by Lars Vogel, you can read this post about it, I came up with a Python version to help me (and hopefully others) publish their update sites to bintray. You can find the script here: py_bintray-publish-p2-updatesite

Using DOT Debug View plugin

As before, all you need to do is provide a method with the signature "String toDOT()" in your class, and return a valid DOT graph string of a representation/view of your object. Of course you need to install Eclipsegraphviz too!.

You can find more details on the project's gitbub page, but basically you can add the following url to your update sites and install it:

https://dl.bintray.com/arcanefoam/DOTDebugView



Monday, July 13, 2015

Using EclipseGraphviz as a debugging aid

My current area of interest is Model Driven Architecture. In this field, and I assume in many others too, our [data] models can be usually represented as graphs. In this scenarios, being able to view our model as an actual graph can be of great help to quickly identify errors in our model.

Come in EclipseGraphviz

Graphviz is a Graph Visualization Software that can produce graph visualizations from a graph representation in a text file (using the DOT language). EclipseGraphviz is a plugin for Eclipse that adds a view in which files with the dot extension will be rendered using Graphviz dot tool.

However, having the need to have a dot file requires you to somehow serialize your model into a file. This limits the use of the viewer for debugging purposes. Surely you can use the console to output a graph in DOT language, then copy the text, paste it to a file... or modify your code to serialize directly into a file...

Wouldn't it be nice to click on a variable in the Variable View during a debug session and have the graph representation of it shown in a nice graph?

EclipseGraphviz + debug

I have extended EclipseGraphviz to make this happen. All you have to do is add a
String toDOT();
method to any class. If the returned string is a valid DOT graph, then you will see it in the View. Simple as that...

This lets you rapidly instrument classes to have a graph representation during debug and doesn't make your projects depend on any of the EclipseGraphviz plugins. It also allows you to keep the toString() method independent if you have a string representation that might be used for other purposes. Having a toDOT() method will also provide you with a means to serialize your model if it needs be.

Bellow you can see a snapshot of how it works. I have selected variable p in the Variables view. The ExecutionPlan class has a toDOT() method. In this case the toString() method just calls toDOT() so you can see the DOT representation of the class in the Variables view.



Installing

The source code is available from GitHub (EclipseGraphviz  fork). I think the easiest way is to follow EclipseGraphviz  installation instructions and then build and export my version to your dropins folder in Eclipse.


Saturday, November 30, 2013

Extended EMC for CVS Files

An extended Epsilon Model Connectivity (EMC) layer for working with Comma-separated Values (CSV) files/models.

Note to the reader: Some of my instructions assume you have worked with the Epsilon languages and tools before.

Epsilon family of languages ad set of tools provide a lot of flexibility when doing al sort of stuff with models. One of the things I like the most is the ability to work with models from domains other than EMF. Epsilon provide support for working with BibTex, CSV, and other type of files/models. Find all the info here. However, some of the drivers only provide basic support. This is the case of the CSV (comma-separated values), which currently only supports loading the CSV model (file) and accessing each of its rows as an element in the model. So basically you where limited to something in the lines of  the following EOL script:

for (r in csv!Row.all()) {
    if (r.at(2)) {
        r.at(0).print();

        r.at(1).print();
        ", is invited to the party!";

    }
}

Which assumes you have a CSV file of the form "name,lastName,isInvited", and you want to print the list of invited guests. Although your EOL script could be more complex you are still limited to read-only operations and index access of fields.

I am currently working on an extended version of the Epsilon's CSV EMC to provide additional capabilities. Essentially I want to provide support to access fields in each row by using the header information as navigation and that also supports modifying the model. You can find the source code in the Epsilon Labs (at Google code). You will need to check out two projects (if you have never worked with SVN repositories in eclipse this link provides all the info you need. Look under installation instructions and how to check out a repository.) : org.eclipse.epsilon.emc.csv and org.eclipse.epsilon.emc.csv.dt, under svnRepo. I will recommend importing this projects to your Eclipse Workspace and then running a nested eclipse to try them out. Replacing your default plugins by this ones might work, but I haven't tested that option. You can also checkout the org.eclipse.epsilon.emc.csv.test project, which has the examples I will be using onwards.

Loading a CSV model

Once the plugins are in place you will be able to use the extended CSV model loader with new options. If you don't see the csv model when adding a model to the lunch configuration be sure to click the "Show all model types" check box.


If you have worked with CSV models before, you will notice that now have a Load/Save section and a new CSV section to set some of the attributes of your CSV model.

  • The Field Separator allows you to select a different separator than comma.... yes, they are called comma-separated files, but sometimes a colon, or a semi-colon, or other char is used as a field separator. Now you can tell the model loader which one too use. By default it is a comma.
  • The Known Headers tells the loader that the first row of your file contains headers. Headers can late be used to access fields of a row.
  • The Varargs Header tells the loader that although the first row has headers, some of the rows in the file may not have values for all of them. I know this is not the "standard" (did you know that RFC 4180 describes CSV file standards?), but my particular CSV files did so I took the liberty to add it.

Lets see how the header information works now inside EOL scripts.

Lets assume I have a CSV model with the following information (some baseball statistics, but I am not a baseball fan):
Rk,Year,Age,Tm,Lg,,W,L,W-L%,G,Finish
1,1978,37,Atlanta Braves,NL,,69,93,.426,162,6
2,1979,38,Atlanta Braves,NL,,66,94,.413,160,6
3,1980,39,Atlanta Braves,NL,,81,80,.503,161,4
4,1981,40,Atlanta Braves,NL,,25,29,.463,55,4
5,1981,40,Atlanta Braves,NL,,25,27,.481,52,5
6,1982,41,Toronto Maple Leafs,AL,,78,84,.481,162,6
7,1983,42,Toronto Maple Leafs,AL,,89,73,.549,162,4
8,1984,43,Toronto Maple Leafs,AL,,89,73,.549,163,2
9,1985,44,Toronto Maple Leafs,AL,,99,62,.615,161,1

If you specify the Known Headers option, then each record (row) of your model will have the attributes Rk, Year, Age, Tm, Lg, etc.. Hence, you can do stuff like this (assuming you named your model baseball):

for (row in baseball!Row.all()) {
    row.Tm.println();
}

I know, its is simple, but enough to show you that now u can access fields in a row by the header name. Pretty neat ah? (You can find this in TestFieldRead.eol)

Saving model modifications:

If you set the "Store on Disposal" value to true, a script like this will change the team name every time u run it. (TestFieldWrite.eol)

for (row in baseball!Row.all()) {
    row.Tm.println();
    if (row.Tm == "Toronto Blue Jays") {
        row.Tm = "Toronto Maple Leafs";
    } else if (row.Tm == "Toronto Maple Leafs") {
        row.Tm = "Toronto Blue Jays";
    }
}

Varargs Headers

The concept of varargs is used in Java to define methods that accept a variable number of parameter (variable arguments). This can happen in a CSV file, where maybe there is a known set of headers that all records have, but also each record may have additional fields beyond this headers. Lets me show this with an example. I do my reference management with Qiqqa, which among other things, lets me export a CSV file of my reference matrix. It looks like this (after some editing to remove the comments and leave only two headers):

source,target
Corradini.etal1996,
Agrawal.etal2006,
DiRuscio.etal2012,Stevens2007,Czarnecki.etal2009,Czarnecki.Helsen2006,Stevens2008,Stevens2010,Selic2003,
Syriani.Vangheluwe2010,Agrawal.etal2006,Jouault.Kurtev2006a,
Aranda.etal2012,France.Rumpe2007,Schmidt2006,Kent2002,
France.Rumpe2007,Czarnecki.Helsen2006,MDA1.0.1,
Schmidt2006,
Kent2002,
Atkinson.Kuhne2003,
Baudry.etal2006,Fleurey.etal2004,
Vallecillo.etal2012,DiRuscio.etal2012,France.Rumpe2007,Czarnecki.Helsen2006,Stevens2008,Baudry.etal2006,

Patzina.Patzina2012,Kolovos.etal2008,Wimmer.etal2011,Jouault.Kurtev2006a,Taentzer.etal2005,Czarnecki.Helsen2006,
Baudry.etal2010,Schmidt2006,France.Rumpe2007,
Guerra2012,Kolovos.etal2008,Baudry.etal2010,
Bezivin2004,OCL2.3.1,Bezivin.Gerbe2001,
Favre2004,MDA1.0.1,Bezivin.Gerbe2001,Bezivin2004,Kent2002,


In such a file, there is always a source filed, but there may be zero or more targets. If you set the Varargs Headers option to true (check it), the CSV model loader will create a record that has a source and a target attribute (or as many fields in your header), and the last header will be a collection. This collection will hold all the fields in each row after and including the last header. IN the previous example, target is the last header, so the header attribute is a collection.

So now, if u want to print a list of all targets of each source, your script may look like this(TestFieldRead.eol) :


for (row in qiqqa!Row.all()) {
    row.source.print();
    " is connected to ".println();
    for (t in row.target) {
        t.println();
    }
}

Header-less CSV files

Finally, for support of header-less CSV files and to make it easy to use your existing scripts, if you don´t check the Known Headers option, the CSV model loader ill create a record for each row and a default attribute field. This attribute is collection which holds all the fields in each row.

Final Notes

Please use the Epsilon forum for any technical questions or discussions :D.

Monday, August 16, 2010

Project Type Module + ChildFactory

After completing the Project Module tutorial (found here), I got a nice folder structure... However version control folders and other files that where not relevant to the project where displayed under the project structure. I wanted to display only the relevant files and folders and hide the rest. Next I show how to build a ChildFactory that does exactly what I needed.

In the original tutorial at Step 2 of section Creating the Logical View Provider, you defined a TextNode that has the following constructor:

public TextNode(Node node, DemoProject project) throws DataObjectNotFoundException {
super(node, new FilterNode.Children(node),
//The projects system wants the project in the Node's lookup.
//NewAction and friends want the original Node's lookup.
//Make a merge of both
new ProxyLookup(new Lookup[]{Lookups.singleton(project),
node.getLookup()
}));
this.project = project;
}

The important bit to notice is the new FilterNode.Children(node) part. This is basically creating a default Children layout for your project. By default I mean that all child files and folders under your main project folder will be displayed under the project node tree. In my specific case the tree I got can be seen in the following figure, where for example you can see version control folders (.svn).




To change this behavior we need to create a new custom child factory. First create a class that extends org.openide.nodes.ChildFactory<T>:

public class AddonChlidFactory extends org.openide.nodes.ChildFactory<String> {

FileObject folder;

public AddonChlidFactory(FileObject folder) {
this.folder = folder;
}
}


The folder FileObject will be used later for inspecting subfolders and filtering them as needed. The ChildFactory works by creating a node for each key in a set of keys (the set of keys is arbitrary). Basically, you supply the set of keys and then the IDE asks you to give him the node(s) that should be generated according to the key. According to the API documentation the set of keys should be generated dynamically (probably according to the actual files and folders in your project folder). But for simplicity I created it statically. My set contains the extensions of the files I am interested and the folders string to provide nodes for folders. This is how my createKeys method looks:

@Override
protected boolean createKeys(List<String> list) {
// File types of interest
list.add("lua");
list.add("toc");
list.add("xml");
list.add("ttf");
list.add("tga");
list.add("blp");
list.add("txt");
list.add("folders");
return true;
}

Next, we need to decide what nodes to return for each key. For data files we will just return the node representing the file. For folders we will inspect the folder name and only return nodes if some conditions are met. Specifically we are interested in NOT returning nodes for version control folders. Next is the code of the createNodesForKey method:

@Override
protected Node[] createNodesForKey(String key) {
ArrayList<Node> nodes = new ArrayList<Node>();
if(key.equals("folders")) {
/* add folder nodes */
for(FileObject o : Collections.list(folder.getFolders(false))) {
if(o.getName().equals("nbproject")) {
try {
nodes.add(new nbProjectNode(o));
} catch (DataObjectNotFoundException ex) {
Exceptions.printStackTrace(ex);
}
} else if(!(o.getName().equals(".svn")
|| o.getName().equals(".cvs")
|| o.getName().equals(".hg"))) {
// Add all folders that are not version control
Node addonFolderNode = DataFolder.findFolder(o).getNodeDelegate();
nodes.add(new AddonFolderNode(addonFolderNode, o));
}
}
} else {
// Add data files
for(FileObject o : Collections.list(folder.getData(false))) {
if (o.hasExt(key)) {
try {
Node addonNode = DataObject.find(o).getNodeDelegate();
nodes.add(new FilterNode(addonNode, Children.LEAF));
} catch (DataObjectNotFoundException ex) {
Exceptions.printStackTrace(ex);
}
}
}
}
return(nodes.toArray(new Node[nodes.size()]));
}

You can notice that for the folders keyword I have two options: First, if the folder is the nbproject folder we create a nbProjectNode (based on the NetBeans Project Type Extension Module Tutorial). Second, if the folder name is not any of the common revision control folder names a node is returned. For data files, we create nodes for any files who's extension is one of the keys. Notice also that for folders we used our own AddonFolderNode node, for which we have a nested class. This will allow us to define custom behavior for our inner project folders (for example what actions can be invoked). Putting it all together we will have:

public class AddonChlidFactory extends org.openide.nodes.ChildFactory<String> {

FileObject folder;

public AddonChlidFactory(FileObject folder) {
this.folder = folder;
}

@Override
protected boolean createKeys(List<string> list) {
// File types of interest
list.add("lua");
list.add("toc");
list.add("xml");
list.add("ttf");
list.add("tga");
list.add("blp");
list.add("txt");
list.add("folders");
return true;
}

@Override
protected Node[] createNodesForKey(String key) {
ArrayList<Node> nodes = new ArrayList<Node>();
if(key.equals("folders")) {
/* add folder nodes */
for(FileObject o : Collections.list(folder.getFolders(false))) {
if(o.getName().equals("nbproject")) {
try {
nodes.add(new nbProjectNode(o));
} catch (DataObjectNotFoundException ex) {
Exceptions.printStackTrace(ex);
}
} else if(!(o.getName().equals(".svn")
|| o.getName().equals(".cvs")
|| o.getName().equals(".hg"))) {
// Add all folders that are not version control
Node addonFolderNode = DataFolder.findFolder(o).getNodeDelegate();
nodes.add(new AddonFolderNode(addonFolderNode, o));
}
}
} else {
// Add data files
for(FileObject o : Collections.list(folder.getData(false))) {
if (o.hasExt(key)) {
try {
Node addonNode = DataObject.find(o).getNodeDelegate();
nodes.add(new FilterNode(addonNode, Children.LEAF));
} catch (DataObjectNotFoundException ex) {
Exceptions.printStackTrace(ex);
}
}
}
}
return(nodes.toArray(new Node[nodes.size()]));
}


private static final class AddonFolderNode extends FilterNode {

FileObject folder;

public AddonFolderNode(Node node, FileObject folder) {
super(node, Children.create(new AddonChlidFactory(folder), true));
this.folder = folder;
}

@Override
public Action[] getActions(boolean arg0) {
Action[] nodeActions = new Action[1];
nodeActions[0] = CommonProjectActions.newFileAction();
return nodeActions;
}

@Override
public String getDisplayName() {
return folder.getName();
}

}

private static final class nbProjectNode extends FilterNode {

public nbProjectNode(FileObject node) throws DataObjectNotFoundException {
super(DataFolder.findFolder(node).getNodeDelegate());
}

@Override
public String getDisplayName() {
return "Important Files";
}
}

}


Finally on our project view we get (look that the svn folder is gone, the nbproject fodler is renamed to Important Files and how files are sorted by type):

Thursday, July 29, 2010

Project Type Module + Privileged Templates

I started working on a Project Module for managing WoW addons (Lua Topping). After fidling arround I got the nodes to work as intended (posted here) I wanted to have a nice set of templates for the basic addon files: lua, toc and xml, plus some specific to WowAce. You can look at the NetBeans tutorial here.

Taking a look at the NetBeans API and the Project class I found that for the lookup you could add a PrivilegedTemplates interface. This interface only has one method:

String[] getPrivilegedTemplates()


According to the API documentation the returned string array should contain "full paths to privileged templates, e.g. Templates/Other/XmlFile.xml". But what is the correct full path for a template?. The simplest way to find out is to view your layer.xml file (you should read about NetBeans and layer files if you want details on how they work). In my case, the Templates section of my layer.xml file looks like this:


<folder name="Templates">
<folder name="WoW">
<file name="TocTemplate.toc" url="TocTemplate.toc">
<attr name="displayName" bundlevalue="net.sourceforge.luatopping.wow.addon.project.Bundle#Templates/WoW/TocTemplate.toc"/>
<attr name="template" boolvalue="true"/>
</file>
<file name="WoWUIxmlTemplate.xml" url="WoWUIxmlTemplate.xml">
<attr name="displayName" bundlevalue="net.sourceforge.luatopping.wow.addon.project.Bundle#Templates/WoW/WoWUIxmlTemplate.xml"/>
<attr name="template" boolvalue="true"/>
</file>
<file name="Template.pkgmeta" url="Template.pkgmeta">
<attr name="displayName" bundlevalue="net.sourceforge.luatopping.wow.addon.project.Bundle#Templates/WoW/Template.pkgmeta"/>
<attr name="template" boolvalue="true"/>
</file>
</folder>
</folder>


It turns out the correct full path for a template is the path starting at Templates/ and finishing at the template name (as in the name property of the file entry), containing all the nested sub folders. So in my case, for the TocTemplate.toc template this would be: Templates/WoW/TocTemplate.toc.

Putting it all together (assuming you have followed the NetBeans tutorial):
  1. Add a PrivilegedTemplates implementation to your project lookup

    //The project type's capabilities are registered in the project's lookup:
    @Override
    public Lookup getLookup() {
    if (lkp == null) {
    lkp = Lookups.fixed(new Object[]{
    state, //allow outside code to mark the project as needing saving
    new ActionProviderImpl(), //Provides standard actions like Build and Clean
    new WowAddonDeleteOperation(),
    new WowAddonCopyOperation(this),
    new Info(), //Project information implementation
    new WowAddonProjectLogicalView(this), //Logical view of project implementation
    new WowAddonProjectCustomizer(this), //Customizer for the project
    new WowAddonProjectPrivilegedTemplates(),
    });
    }
    return lkp;
    }


  2. In you PrivilegedTemplates implementation construct the desired templates names array

    private final class WowAddonProjectPrivilegedTemplates implements PrivilegedTemplates {

    private String[] privileged = new String[]{
    "Templates/WoW/TocTemplate.toc",
    "Templates/Other/LuaTemplate.lua",
    "Templates/WoW/WoWUIxmlTemplate.xml",
    "Templates/WoW/Template.pkgmeta",
    };

    public String[] getPrivilegedTemplates() {
    return privileged;
    }
    }



That's it!. When you run your Project Module, your templates should be visible when you right click the project node and select the New option: