Status Report: December 2017

Well, another month and year has flown by. I hope that everyone had a very happy holiday season, and here’s wishing you all a fun and profitable new year!

What I worked on this month:

– I’m still working on getting the debugger to start up reliably for Flash Debugging. I was working on getting the file naming right for the .SWF files when running and debugging because there are several wrong solutions in place. Well, that didn’t get completed, and while thinking about things, I thought that there might be a better approach, which I’m experimenting with at the moment. I also realized that when I broke up the lime commands from “lime test” into “lime build” and “lime run”, there is a “lime install” step missing. I’m addressing that last point at the moment, then I’ll probably get a test build out, and then I’ll get back to getting .SWF files to run reliably.

– There was a huge community submission adding new language model classes, laying the foundation for a resolver rewrite.  Roughly 6000 lines of code were added and about 4000 removed.  That review took a week all by itself!  I sure appreciate the help, though.  That code isn’t in the mainline yet; there are a few changes that we are still discussing.

Again, Happy New Year!

Status Reports?

Every month, I spend a few minutes writing progress or status reports for my corporate sponsors (for which I am truly thankful).  Frequently, these are related to efforts on the IntelliJ-Haxe plugin.  They are pretty simple stuff, usually.  A bunch of one-liner summaries so that folks know what I and others in the community have been working on.  I’ll add a bit of explanation when it’s useful (like why a task is taking more time than I would expect, or what underlying issues were exposed by making related changes).  And I got to thinking…  Why not just post those reports to the blog?  Maybe you folks would like to read about recent progress?  Well, on the premise that some of you actually care, I’m going to do just that.

Here are the status reports for the last several months (in reverse chronological order):

November 2017:

What I worked on this month:

  • Split the build and run phases for debugging Flash targets. Found underlying issues that require a new solution.
  • 0.11.2 Release.
  • Fixed 16 classes of parsing errors. (The last four after the release.)
  • Wrote a ton of parsing tests.
  • NPE fix for haxelibs without metadata.
  • Disabled a bunch of debug logging that was filling idea.log.

Community contributions:

  • Added Adobe AIR target.
  • Fixed a document link.
  • Fixed a couple of null references in the lime and hxml project file models.

October 2017:

This month a client had a request to complete a task that was something another client had been asking for.  The task was to “Include all implementations of the interface in usage search.” In other words, prompt during FindUsages if a method that was being searched for was declared in a superclass or interface, whether to search for callers of the super-method or just those of the current class.  That is now complete and has been made available in a preview build (0.11.2.Preview.4).  That will go live very shortly.

What I did this month:

  • Implemented “Include all implementations of interface in usage search.”
  • Fixed parsing block statements when used as an intializer. For example:
      class Test {
        private static var a = { // <----- Used to have a parsing error here.
          var t = "Yadayada";
          t;
        } // <---- And another here.
      }
    
  • Fixed exceptions when running haxelib and the SDK is incorrectly configured.
  • Refactored automatic haxelib library dependency detection and project synchronization. It now follows the complete (multi-level) dependency graph and runs much faster.
  • Add a UI option to disable haxelib library dependency synchronization at the module level.
  • 0.11.2.Preview.4 drop.

There were no contributions from the community this month.

September 2017:

Here’s the work I did:

  • Spent my time doing more work to make flash debugging reliably launch the .SWF. To that end, I added code to:
    • parse the lime-produced debug/release.hxml files to ensure that I could find the proper .SWF and runtime args;
    • invalidate and track project and lime output changes;
    • added the haxe and neko directories to the run-time path;
  • UI work to support remapping of sources from one library to another without changing import lines. (e.g. –remap support). Underlying support in the resolver is not written yet.
  • Rewrite of the module/haxelib dependency gathering (auto-updating the dependency tree for haxelibs). This helps resolving variables when one haxelib depends upon another and the user is unaware of why “find usages” and “goto” don’t seem to be working correctly.

Community contributions:

  • Fixed Windows build environment for the plugin.
  • Fix for keeping environment variables that were being dropped in some cases. (Depending upon which internal process launcher was used.)
  • Fixed NPE when static methods in statically imported classes had no parameters.
  • Fixed parameter count resolving for anonymous functions.
  • Better parser recovery after “extends” and “implements” statements.
  • Fixed parser error when dot-separated types are used in type parameters (“generics”).

August 2017:

This month I spent time on the following:

Getting flash (.swf) file debugger starting the debugger properly. I found one of the underlying problems is the environment that is in place when IDEA starts up. Unfortunately, it is in IDEA’s code and not something that can be worked around with code in the plugin. I sent an email separately talking about the issues and the workaround. There is still work to be done and I’ve been working through it.

  • Getting the hxcpp debugger to display the proper source file when source files do not properly live in either the compiled tree or the project subtrees (meaning they are referenced as symlinks to another projects source tree).
  • Adding –remap support (e.g. flash -> openFL) within the plugin. This work is to help Find, Goto References, and type resolution to be more correct. (And when type resolution is better, lots of other things magically work better.)

 

There were no community submissions this month.

Just what is a “selectioner” anyway?

Getting double-clicks to highlight what you want.

I’ve just spent a day and a half fixing bug #212, where the IntelliJ-Haxe plug-in doesn’t appropriately select words inside of string constants (a.k.a. characters enclosed in quotes: “This is a string.”).  Before I fixed it, when you double-clicked on a string constant in the editor, it selected the entire string rather than the word you clicked on.  Most programmers expect the word to be selected and get frustrated.

It turns out that while I was fixing the string handling that the comment handling was broken, too, as seen in this animation:

So, with a little experimentation and a little extra effort, I was able to correct both issues:

Behavior of double-clicking in comments and string constants.

Hopefully, folks will find their double-clicking a bit more predictable.

How it’s done.

From here on in, this post gets quite technical, and is really targeted to plug-in authors. As such it assumes general knowledge of JetBrains’ plug-in architecture.

Now, JetBrains’ openapi, which IDEA uses extensively, has a default action for selection that handles a lot of general cases that are good enough for most programming languages.  Selections are handled by an action handler, and we can override that just by providing an implementation and noting that in the plug-in’s xml descriptor file (we’ll get there in a minute).  However, rewriting the entire action means that the original action selection handlers will no longer be available and/or we would have to duplicate a lot of code.  There’s got to be a better way…

It turns out that the selection action itself uses pluggable “selectioners” to handle the actual selection process.  (While I don’t like the name JetBrains chose, I guess it is easier to type than “SelectionProcessor,” “SelectionCreator,” “SelectionHandler” etc.)  These are much easier to create and they don’t require rewriting the entire selection mechanism.  Here is the API:


public interface ExtendWordSelectionHandler {
  ExtensionPointName EP_NAME = ExtensionPointName.create("com.intellij.extendWordSelectionHandler");
  boolean canSelect(PsiElement e);
  List select(PsiElement e, CharSequence editorText, int cursorOffset, Editor editor);
}

The first member is the name of the extension point, which we will use in the plugin’s xml descriptor (src/META-INF/plugin.xml in our tree). The second member, the canSelect method, determines whether this handler can create a selection using given PSI element. The third member, select, actually calculates the selection.

When IDEA determines that it’s time to do a word selection, it goes down the list of *ALL* installed selection handlers.  It calls each of their canSelect methods and remembers which of them had positive responses.  Then, it calls the select method on each of those, and compares each of their results.  The result with the smallest specific selection is kept.  The code is in com.intellij.codeInsight.editorActions.SelectWordUtil.processElement(). Basically: (code paraphrased for clarity)


    for (ExtendWordSelectionHandler selectioner : extendWordSelectionHandlers) {
      if (selectioner.canSelect(element)) {
        availableSelectioners.add(selectioner);
      }
    }
    for (ExtendWordSelectionHandler selectioner : availableSelectioners) {
      List ranges = selectioner.select(element, text, cursorOffset, editor);
      for (TextRange range : ranges) {
        if (minimumRange.get().contains(range)) {
          minimumRange.set(range);
        }
      }
    }

What this means is two-fold: It is up to the handler to decide whether it can work with the type, but it CANNOT guarantee that it is the only handler that will create a selection. Nor can it guarantee that its result will be the one used; the result competes with all others to be the most specific selection. If you want to fully control the selection process, you will have to override the editor action.

At any rate, you can see our solution in the github repository. As you can see, we first be sure that our Haxe-specific handler will not interfere with other languages:


  public boolean canSelect(PsiElement e) {
    return (e instanceof HaxePsiToken   // <---- This line implies that it's a Haxe language token.
            && e.getLanguage().equals(HaxeLanguage.INSTANCE)  // <---- This line ensures that another language hasn't subclassed our type.
            && HaxeTokenTypes.REGULAR_STRING_PART.equals(((HaxePsiToken)e).getTokenType()));  // <---- And this one ensures that we're only working with strings.
}

Then we implement the actual selection algorithm. I don’t duplicate that code because it’s a bit lengthy, but you can find it here. It might be easier to look at it in the Pull Request. Then you’ll be able to see the code used to determine the bounds of the word.

At any rate, we are given the position of the cursor, the PSI element at that position, the (entire) text of the buffer in the editor, and a pointer to the editor window itself. From these, we start at the given position, get the text for the element, and walk backward and forward looking for word termination character. If the position is on a space and a word ends on the prior position, then we’ll select that word.

The last thing that we have to do is register the new handler. This is done in the plugin.xml file. We only had to specify the extension point name and the implementing class:


   <extensions defaultExtensionNs="com.intellij">
    [...]
    <extendWordSelectionHandler implementation="com.intellij.plugins.haxe.editor.actions.wordSelection.HaxeStringSelectioner" />
    [...]
   </extensions>

Note that the entire name for the extension point as given in the API is com.intellij.extendWordSelectionHandler. However, the defaultExtensionNs does not have to be respecified on the actual implementation line. (However, if you place the directive in an extensions section that uses a different default, then you need to specify the whole string. I haven’t tried this, myself.)