Status Report: December 2017

Well, another month and year has flown by. I hope that everyone had a very happy holiday season, and here’s wishing you all a fun and profitable new year!

What I worked on this month:

– I’m still working on getting the debugger to start up reliably for Flash Debugging. I was working on getting the file naming right for the .SWF files when running and debugging because there are several wrong solutions in place. Well, that didn’t get completed, and while thinking about things, I thought that there might be a better approach, which I’m experimenting with at the moment. I also realized that when I broke up the lime commands from “lime test” into “lime build” and “lime run”, there is a “lime install” step missing. I’m addressing that last point at the moment, then I’ll probably get a test build out, and then I’ll get back to getting .SWF files to run reliably.

– There was a huge community submission adding new language model classes, laying the foundation for a resolver rewrite.  Roughly 6000 lines of code were added and about 4000 removed.  That review took a week all by itself!  I sure appreciate the help, though.  That code isn’t in the mainline yet; there are a few changes that we are still discussing.

Again, Happy New Year!

Status Reports?

Every month, I spend a few minutes writing progress or status reports for my corporate sponsors (for which I am truly thankful).  Frequently, these are related to efforts on the IntelliJ-Haxe plugin.  They are pretty simple stuff, usually.  A bunch of one-liner summaries so that folks know what I and others in the community have been working on.  I’ll add a bit of explanation when it’s useful (like why a task is taking more time than I would expect, or what underlying issues were exposed by making related changes).  And I got to thinking…  Why not just post those reports to the blog?  Maybe you folks would like to read about recent progress?  Well, on the premise that some of you actually care, I’m going to do just that.

Here are the status reports for the last several months (in reverse chronological order):

November 2017:

What I worked on this month:

  • Split the build and run phases for debugging Flash targets. Found underlying issues that require a new solution.
  • 0.11.2 Release.
  • Fixed 16 classes of parsing errors. (The last four after the release.)
  • Wrote a ton of parsing tests.
  • NPE fix for haxelibs without metadata.
  • Disabled a bunch of debug logging that was filling idea.log.

Community contributions:

  • Added Adobe AIR target.
  • Fixed a document link.
  • Fixed a couple of null references in the lime and hxml project file models.

October 2017:

This month a client had a request to complete a task that was something another client had been asking for.  The task was to “Include all implementations of the interface in usage search.” In other words, prompt during FindUsages if a method that was being searched for was declared in a superclass or interface, whether to search for callers of the super-method or just those of the current class.  That is now complete and has been made available in a preview build (0.11.2.Preview.4).  That will go live very shortly.

What I did this month:

  • Implemented “Include all implementations of interface in usage search.”
  • Fixed parsing block statements when used as an intializer. For example:
      class Test {
        private static var a = { // <----- Used to have a parsing error here.
          var t = "Yadayada";
          t;
        } // <---- And another here.
      }
    
  • Fixed exceptions when running haxelib and the SDK is incorrectly configured.
  • Refactored automatic haxelib library dependency detection and project synchronization. It now follows the complete (multi-level) dependency graph and runs much faster.
  • Add a UI option to disable haxelib library dependency synchronization at the module level.
  • 0.11.2.Preview.4 drop.

There were no contributions from the community this month.

September 2017:

Here’s the work I did:

  • Spent my time doing more work to make flash debugging reliably launch the .SWF. To that end, I added code to:
    • parse the lime-produced debug/release.hxml files to ensure that I could find the proper .SWF and runtime args;
    • invalidate and track project and lime output changes;
    • added the haxe and neko directories to the run-time path;
  • UI work to support remapping of sources from one library to another without changing import lines. (e.g. –remap support). Underlying support in the resolver is not written yet.
  • Rewrite of the module/haxelib dependency gathering (auto-updating the dependency tree for haxelibs). This helps resolving variables when one haxelib depends upon another and the user is unaware of why “find usages” and “goto” don’t seem to be working correctly.

Community contributions:

  • Fixed Windows build environment for the plugin.
  • Fix for keeping environment variables that were being dropped in some cases. (Depending upon which internal process launcher was used.)
  • Fixed NPE when static methods in statically imported classes had no parameters.
  • Fixed parameter count resolving for anonymous functions.
  • Better parser recovery after “extends” and “implements” statements.
  • Fixed parser error when dot-separated types are used in type parameters (“generics”).

August 2017:

This month I spent time on the following:

Getting flash (.swf) file debugger starting the debugger properly. I found one of the underlying problems is the environment that is in place when IDEA starts up. Unfortunately, it is in IDEA’s code and not something that can be worked around with code in the plugin. I sent an email separately talking about the issues and the workaround. There is still work to be done and I’ve been working through it.

  • Getting the hxcpp debugger to display the proper source file when source files do not properly live in either the compiled tree or the project subtrees (meaning they are referenced as symlinks to another projects source tree).
  • Adding –remap support (e.g. flash -> openFL) within the plugin. This work is to help Find, Goto References, and type resolution to be more correct. (And when type resolution is better, lots of other things magically work better.)

 

There were no community submissions this month.

Release 0.11.2 is now available.

We are proud to release our 0.11.2 release of the IntelliJ-Haxe plugin.  We’ve been hard at work fixing parsing issues and adding new usability features.  Chief among the latter is that “Find Usages” now presents the option to search for usages of a parent interface method instead of just the instance under the cursor.  You’ll also find that the automatic project updates to haxelib dependencies has been rewritten — and can be disabled via the user interface.  Now, it traverses the entire dependency tree and adds all of the libraries that the compiler would.  It’s also been made a LOT faster.

In related news: The Haxe Foundation’s fork of the plugin’s sources have now become the primary repository; JetBrains made their repository a fork of ours!  This is great news because it means that searches for Haxe and IntelliJ on github (and other search engines!) now point to our project rather than the inactive sources that JetBrains had originally provided.

The full sources and associated release notes can be found at github: https://github.com/HaxeFoundation/intellij-haxe/releases/tag/0.11.2

The plugin can be downloaded within IntelliJ-IDEA by following the installation instructions here.

As always… Enjoy!

-Eric

 

Release 0.11.1 is now available.

Hi folks,

Today, we released version 0.11.1 of the IntelliJ-Haxe plugin.  You can pick it up in JetBrains plugin repository, or on github.

This is primarily a bug-fix version, but you will likely be happy about seeing these fixed:

  • Check for and halt type resolution when a cyclical/recursive definition is found.
  • Address some freezes by delaying use of indices until indexing is complete.
  • Speed haxelib syncing (and stop unnecessary re-indexing). (Regression)
  • Fix freezes by fixing some multi-threading issues and other exceptions being thrown.
  • Speed up parsing of arrow functions.
  • Add Haxe-specific double-click selection logic for strings and comments. (Issue #212)
  • Reroute debugging informational errors to the status bar instead of modal dialogs.
  • Fix compilation halting on “- Link” informational messages.
  • Add neko and haxelib directories to the path when building projects (for all platforms; used to be OSX-only).
  • Fix multi-platform build issues (for the plugin, particularly affects Windows builds).

Take a test drive and tell us what you think! We love to hear from you.

Just what is a “selectioner” anyway?

Getting double-clicks to highlight what you want.

I’ve just spent a day and a half fixing bug #212, where the IntelliJ-Haxe plug-in doesn’t appropriately select words inside of string constants (a.k.a. characters enclosed in quotes: “This is a string.”).  Before I fixed it, when you double-clicked on a string constant in the editor, it selected the entire string rather than the word you clicked on.  Most programmers expect the word to be selected and get frustrated.

It turns out that while I was fixing the string handling that the comment handling was broken, too, as seen in this animation:

So, with a little experimentation and a little extra effort, I was able to correct both issues:

Behavior of double-clicking in comments and string constants.

Hopefully, folks will find their double-clicking a bit more predictable.

How it’s done.

From here on in, this post gets quite technical, and is really targeted to plug-in authors. As such it assumes general knowledge of JetBrains’ plug-in architecture.

Now, JetBrains’ openapi, which IDEA uses extensively, has a default action for selection that handles a lot of general cases that are good enough for most programming languages.  Selections are handled by an action handler, and we can override that just by providing an implementation and noting that in the plug-in’s xml descriptor file (we’ll get there in a minute).  However, rewriting the entire action means that the original action selection handlers will no longer be available and/or we would have to duplicate a lot of code.  There’s got to be a better way…

It turns out that the selection action itself uses pluggable “selectioners” to handle the actual selection process.  (While I don’t like the name JetBrains chose, I guess it is easier to type than “SelectionProcessor,” “SelectionCreator,” “SelectionHandler” etc.)  These are much easier to create and they don’t require rewriting the entire selection mechanism.  Here is the API:


public interface ExtendWordSelectionHandler {
  ExtensionPointName EP_NAME = ExtensionPointName.create("com.intellij.extendWordSelectionHandler");
  boolean canSelect(PsiElement e);
  List select(PsiElement e, CharSequence editorText, int cursorOffset, Editor editor);
}

The first member is the name of the extension point, which we will use in the plugin’s xml descriptor (src/META-INF/plugin.xml in our tree). The second member, the canSelect method, determines whether this handler can create a selection using given PSI element. The third member, select, actually calculates the selection.

When IDEA determines that it’s time to do a word selection, it goes down the list of *ALL* installed selection handlers.  It calls each of their canSelect methods and remembers which of them had positive responses.  Then, it calls the select method on each of those, and compares each of their results.  The result with the smallest specific selection is kept.  The code is in com.intellij.codeInsight.editorActions.SelectWordUtil.processElement(). Basically: (code paraphrased for clarity)


    for (ExtendWordSelectionHandler selectioner : extendWordSelectionHandlers) {
      if (selectioner.canSelect(element)) {
        availableSelectioners.add(selectioner);
      }
    }
    for (ExtendWordSelectionHandler selectioner : availableSelectioners) {
      List ranges = selectioner.select(element, text, cursorOffset, editor);
      for (TextRange range : ranges) {
        if (minimumRange.get().contains(range)) {
          minimumRange.set(range);
        }
      }
    }

What this means is two-fold: It is up to the handler to decide whether it can work with the type, but it CANNOT guarantee that it is the only handler that will create a selection. Nor can it guarantee that its result will be the one used; the result competes with all others to be the most specific selection. If you want to fully control the selection process, you will have to override the editor action.

At any rate, you can see our solution in the github repository. As you can see, we first be sure that our Haxe-specific handler will not interfere with other languages:


  public boolean canSelect(PsiElement e) {
    return (e instanceof HaxePsiToken   // <---- This line implies that it's a Haxe language token.
            && e.getLanguage().equals(HaxeLanguage.INSTANCE)  // <---- This line ensures that another language hasn't subclassed our type.
            && HaxeTokenTypes.REGULAR_STRING_PART.equals(((HaxePsiToken)e).getTokenType()));  // <---- And this one ensures that we're only working with strings.
}

Then we implement the actual selection algorithm. I don’t duplicate that code because it’s a bit lengthy, but you can find it here. It might be easier to look at it in the Pull Request. Then you’ll be able to see the code used to determine the bounds of the word.

At any rate, we are given the position of the cursor, the PSI element at that position, the (entire) text of the buffer in the editor, and a pointer to the editor window itself. From these, we start at the given position, get the text for the element, and walk backward and forward looking for word termination character. If the position is on a space and a word ends on the prior position, then we’ll select that word.

The last thing that we have to do is register the new handler. This is done in the plugin.xml file. We only had to specify the extension point name and the implementing class:


   <extensions defaultExtensionNs="com.intellij">
    [...]
    <extendWordSelectionHandler implementation="com.intellij.plugins.haxe.editor.actions.wordSelection.HaxeStringSelectioner" />
    [...]
   </extensions>

Note that the entire name for the extension point as given in the API is com.intellij.extendWordSelectionHandler. However, the defaultExtensionNs does not have to be respecified on the actual implementation line. (However, if you place the directive in an extensions section that uses a different default, then you need to specify the whole string. I haven’t tried this, myself.)

Release 0.11.0 is out.

Hello Folks!

We’ve got a new release for you, with a more stable product and improved usability.  Take a look at our notable changes:

  • Support IDEA 2017.1
  • Include completions from the compiler. (Better typing — and results when the plug-in can’t figure things out.)
  • Better handling of type parameters, including following types through multiple levels.
  • Correct parsing of compiler conditionals; allows for more complete typing information.
  • Better ability to resolve types. (Including inside of loops.)
  • Better handling of constructors, including parameter type hints (tooltips).

Give this release a spin and tell us what you think!  We would love to hear from you.

 

The full change log:

0.11.0: (community release)

  • Support IDEA 2017.1
  • Add parsing support for “Arrow Functions.”
  • Better recovery of parsing errors in function parameter lists.
  • Fixed exceptions occurring when adding libraries, so auto-adding will work again.
  • Delay using project indexes until scanning is complete.
  • Proper resolution of constructors (‘new’).
  • Display parameter tip text when creating new object instantiations.
  • Better parsing of shift-and-assign operators.
  • Now correctly resolves variables declared in ‘for’ statements when the iterated type is parameterized. (Issue #528)
  • Resolve chained classes with type parameters (generics).
  • Correct completion with EitherType<>. (Issue#512).
  • Parse @:const type parameters without error. Also allow constants as type parameters.
  • Added navigation to getter/setter methods from property accessors.
  • Annotate strings with incorrect quotes and add quick-fix intention to convert them.
  • Note optional arguments with a ? when displaying methods.
  • Improved method signature check.
  • Added searching of implementation declared by superclasses.
  • Properly parse and evaluate compiler conditionals (#if…#else…#end)
  • Resolve array access with types other than “Array.”
  • Better ‘Main class’ chooser for the ‘Project Settings->Haxe Compiler’ dialog.
  • Fix property getter/setter quick-fixes.
  • Add location data, if known, to compiler completion error messages.
  • Display available completions even when the compiler reports an error.
  • Fixed incomplete results from a compiler run.
  • Better logic for removing duplicate entries from completion lists.
  • Better code completion using the compiler — OFF BY DEFAULT! Turn on in File->Project Structure…
  • Fix parsing of all compiler conditionals. (#417, #121, partly #115, and others)
  • Fix parsing of one-liner conditional compilation style (issue #417, #121, partly #115)
  • Support for `@:require` haxe_ver comparing (issue #418)
  • Support for `@:require` and `@:jsRequire` with multiple arguments
  • Better handling of closing parens, brackets, quotes. (Issues #545, 546)
  • Fix parsing when an anonymous function call is defined and immediately executed. (Issue #544)
  • Fix library name parsing issues for haxelibs using non-standard paths.
  • Resolve URLs properly when adding haxelibs.
  • Updated Haxe logo bitmaps.