Author: Eric Bishton

Hi folks,

A death by a thousand cuts. That’s what the CPU overuse problem really amounted to. By finding a few files that produced the problem, and then spending quite a bit of time in the YourKit profiler, I was able to find a number of places that created small delays, each relatively inconsequential in themselves. As I was attacking each of them, I kept wondering, “Is this the big issue? Is this the smoking gun?” Well, yes and no, as it turns out.

The primary issue was that resolution was slow. It was slow because it was re-doing a lot of work. Adding caching was a help, but no magic bullet. Parsing, itself, was still too slow. In the end, the parsing culprits turned out to be String.split(), the streaming Collections.collect() interface, and parsing float values — all Java intrinsic functions. The answer turned out to be: don’t do those things. Replacing and/or avoiding each of them saved time incrementally.

As it stands, parsing and annotation processing are about ten times as fast as they were. That leaves more time in the UI thread so that colorizing and annotating the sources no longer appear to delay user’s typing response.

What I did for my sponsors:
1) CPU Over-use bug. Profiling, code path optimization. (Complete? In testing.) (Also fixes bug #851)

What I did for the Community:
1) Small number of code reviews.
2) Some work on fixing the debugger launching the wrong configuration when HXML or custom configurations are used.

Community Contributions (Thanks to Alexander Kuzmenko):
1) Auto-setup of the Haxe SDK for new projects.
2) Auto-setup of the Haxe SDK for projects imported from source.
3) Auto-setup of libraries and run configuration when importing from sources.

Next Month:
1) Resolve overlap between my Haxe SDK work and Alexander’s; finish new UI for standard library locations.
2) Release 11.1 (not 11.3 — we are currently at 11.0.2)
3) (Stretch) More work toward parsing and syncing project files, leading to the automatic managing for compiler conditional problems.

-Eric

Hi folks,

This month, most of my time went to the CPU over-use issue. Thanks to a bug report from Matan Uberstein, I have finally been able to duplicate it on a Windows PC, too. The base problem is that “resolving” (or figuring out what/where the type that is being referred to is defined) has slowed down and it is most apparent when working with large files — either editing or including them. This was due to a restructuring of that code: refactoring to use our model APIs and making it compliant with the Haxe language (if multiple versions of a name were found, we may have picked the wrong one). What I said last month is still true: the background processing of annotations (“Find Unused Imports” and colorizing) is much slower, and that is in large part due to the resolution issue. I have some more optimizations and I’ll keep working on it.

What I did for our sponsors:
1) CPU Over-use bug. Profiling, code path optimization.

What I did for the Community:
1) Small number of code reviews.
2) Worked on using HAXE_STD_LIB environment variable and associated UI and project changes (ongoing).

Contributions from the community:
1) Initial “Hello World!” content for new projects.  (Thanks to Alexander Kuzmenko!)

Next month [same as last month 🙁 ]:
– CPU Over-use: Finish optimizing and/or refactor “Find Unused Imports” inspection.
– Finish using HAXE_STD_LIB environment variable and be able to set it inside of IDEA.
– Fix bug where Neko (NME) is trying to be launched when custom or HXML configurations are specified.
Stretch goal – Release 11.3
Stretch goal – More work toward parsing and syncing project files, leading to the automatic managing for compiler conditional definitions.

Hi Folks,

June was kind of a slow month, results-wise, but plenty of stuff got worked on…

For my sponsors, most of my time went to dealing with the CPU Over-use issue, where the CPU is running at roughly 300% forever on a Mac.  I’ve been having trouble duplicating the issue exactly.  However, we were able to create a similar results by opening up a specific file, approximately 1000 lines long, with a few dozen imports (including OpenFL). I say “similar” because it’s not clear that it’s the same issue, in that the CPU runs high for a few minutes and then calms back down if no editing occurs. We agree that it *might* be the issue, so I’m addressing it.

In that vein, we found that the problem is the background annotations; specifically, the “Find Unused Imports” inspection. In profiling that code, I have found several hot-spots and have fixed a few of them, bringing the total run time down about 15%. That’s not enough to deal with the problem, though, and the other hot spots aren’t accessible (they are in IDEA’s code — accessing the file caches). Therefore, I’m looking to change the algorithm so that we don’t have to call into IDEA’s code so much.

As for other things that got done over the month, there have not been many checkins, but things have been happening…

What I did for our sponsors:
1) CPU Over-use bug.
1a) Duplicate the sponsor’s code environment on site.
1b) Also set it up for plugin development, so that I can debug and test changes in situ.
1c) Duplicate and profile the bug.
1d) Optimize critical code path, resulted in 15% run time reduction. (Continuing)

What I did for the Community:
1) Code review of Ilya’s latest semantic annotation rewrite — needs changes.
2) Code review for Mads’ build changes (using gradle) — not yet merged, but approved.
3) Checked in the example project showing hxcpp target debugging with openfl.
4) Worked on using HAXE_STD_LIB environment variable and associated UI and project changes (ongoing).

Contributions from the community:
1) Mads’ changes to use gradle instead of ant to build the plugin are complete, but not yet merged. (Waiting for the CPU Over-use bug to be finished; it has higher priority and changing to gradle may be disrupting to work flows.)

Next month:
– CPU Over-use: Finish optimizing and/or refactor “Find Unused Imports” inspection.
– Finish using HAXE_STD_LIB environment variable and be able to set it inside of IDEA.
– Fix bug where Neko (NME) is trying to be launched when custom or HXML configurations are specified.
– Release 11.3 (Stretch goal)
– More work toward parsing and syncing project files, leading to the automatic managing for compiler conditional definitions. (Stretch goal)

Hi folks,

May was an exciting month, what with the trip to the Haxe Summit this year. I was glad to meet with you and make the acquaintance of folks I’ve only known by name, thus far.

While I was there, I took the opportunity to spend time with our sponsors, Flowplay in their Seattle office, I spent a bit of time with Mark Aoyama, who mentioned a couple of frustrations he was having. It turned out that he was having speed issues that were due to another couple of plugins — that he doesn’t use — generating NPEs. When we removed them, his life was suddenly much better.  So, let that be a reminder to us all: don’t run plugins you don’t need; they slow down the overall IDEA environment and can cause unexpected issues.

Mark also pointed out that he would really like type completion when creating new methods. It was frustrating to him that completion wasn’t triggered by a ‘:’. He added it to the list of things to work on, and now it’s implemented. It is currently available in the new EAP build and will be in the next release which should come very soon.

Speaking of the EAP build, while we were all enjoying the spring weather in Seattle, Ilya Malanin was hard at work creating a new nightly build for the plugin (built against the latest *released* IDEA).  It is released onto the JetBrains’ plugin repository in the “EAP” channel.  You can enable it (and get notifications of new builds) inside of IDEA by following the directions for adding custom repositories.  Simply downloading it from the repository and installing from disk is useful, too, if you don’t like the integration.

Things I did for sponsors this month:
– Worked on CPU over-use issue on Mac.
– Added ‘:’ to the list of triggers when types are expected. (Context sensitive; not in select statements.) (Available in EAP build.)

Things I did for the community this month:
– Added UI for the location of the Haxe std library, instead of assuming it’s on the classpath. This is to eliminate a set of bugs, most notably on the Mac, where std library classes cannot be found. We still have to *use* this value when setting up SDKs, and it’s not yet merged into the development branch.
– Created an example/sample OpenFL project with a C++ target, primarily as a demonstration of hxcpp debugging. (Not yet merged into the main branch.)
– The usual batch of code reviews.

Community contributions
From Ilya Malanin:
– Nightly developer builds (from the “develop” branch) published to the EAP channel on the JetBrains plugin repository.
(https://plugins.jetbrains.com/plugin/6873-haxe-toolkit-support)
– Fixed “Find Usages” for properties.
– Support of “Find Usages” for constructors.
– Fully re-written semantic annotations, added a lot of new checks and quick fixes. Not yet merged.
– Fixed indents for fat arrow expressions.
– Fixed indents for anonymous structures fields and extends lists.
– Fixed NPE that occurred if current project SDK was not properly configured.
– Fixed parsing of anonymous type with empty body.
– Add anonymous types in anonymous types support
– Add anonymous structures names support
– Added chained anonymous fields recognition
– Add proper generics propagation for anonymous structures, typedefs and classes
– Add generic constraint support
– Add Null<T> support
– Add proper from-to declaration support, types infer and compatibility checks
– Fix referenced var-init support check

From Mads Lundemo:
– Gradle as our primary build environment for the plugin. Not yet merged into development.

Next month:
– Finish using HAXE_STD_LIB environment variable and be able to set it inside of IDEA.
– Fix bug where Neko (NME) is trying to be launched when custom or HXML configurations are specified.
– Review Ilya’s latest semantic annotation (5000 line+) rewrite.
– Release 11.3
– More work toward parsing and syncing project files, leading to the automatic managing for compiler conditional definitions.

This month I got the release out that I had been promising for a couple of months. We decided to change the numbering to 1.x from the previous 0.x release numbering for two reasons: One, we’ve broken backward compatibility with any pre-Java8 products — IDEA 15.x and earlier; the second is that the product has been past the minimum viable product stage for a few years now, and there is no reason to continue to make people think that it’s not ready for prime time. That’s not to say that the product is full-featured, or that we’re done by any means.

The folks over at FlowPlay are seeing some performance issues with the new release.  Firstly, a high-CPU usage on macOS, which I haven’t been able to duplicate yet.  Secondly, overall slowdown when typing, with the engineers being able to type several characters ahead of what is displayed on the Windows platform.  I have spent some time on the issue and have some leads on the Windows side, but what I have found is inconsistent.  I’ll keep looking.  In the mean time, if others are seeing the issue, let us know what your setup is (e.g. platform, versions, etc.).

I believe that Flash Debugging is working properly now, but I need you folks to test it and let me know if it’s working for you. The last public release has all of my changes addressing the issue. I can reliably start the AIR debugger *as long as* the debugger is associated with .SWF files at the OS level. Another way to think about it is that simply typing the .SWF name on the command line should launch the app in the AIR player. If it does, then OpenFL — and IDEA for non-OpenFL setups — can run the application; if it does not, then neither can and the AIR debugger must be associated with the .SWF file extension.

What I did this month for my sponsors:
– Flash debugging fixes:
o Projects now use a better algorithm to find and start the built .SWF files.
o Unified calculations of run target names.
– Map Initializers
o Parse map initializers as their own entity instead of treating them like arrays.
o Add new rules for type checking of map initializers. (Removed error annotations.)
– CPU Over-use resolution (incomplete)
o Found Java8 issue on Windows.

Other things I worked on:
– Add local variable type checking (for error annotations).
– Improve array type checking.
– Support arrow functions when evaluating types.
– Infer function typing from code blocks.
– Add $trace() to the list of built-in functions to recognize.
– Made a release.

Community Contributions (mostly @mayakwd):
– Use class paths as sources for the standard lib.
– Validate SDK entries
– Help from @mayakwd finding and fixing an NPE when using “dev” versions of haxelibs.

Next Month:
– Duplicate and fix the CPU over-use issue on macOS.
– 1.0.1 Point release!
– IDEA 2017.3 and 2018.1 support and bugs fixed.
– More work toward parsing and syncing project files, leading to the automatic managing of compiler conditional definitions.

Have a great month!
-Eric

Hi folks,

I did lots of things this last month, but for a number of reasons I did not get a release out the door. That’s my first goal, but it requires that I get Flash debugging working again for projects that are not OpenFL-based. In my attempt to fix debugging for OpenFL last month, I broke it in other ways. Once that’s fixed, I’ll get the next release out and get moving on other things.

Speaking of that feature-test build I made last month, did anybody test it? I haven’t heard anything, so “No news is good news,” I guess.

For my sponsors:
– More work on collapsible regions
– Fixes to error message parsing.
– Proper Map initialization. (e.g. Understand [ “one” => 1, “two” => 2] as a Map instead of an Array.)
– More changes to Flash Debugging. (Incomplete)

Other things I worked on:
– Added C#-style regions to recognized region folding markers.
– Fixed a couple of NPEs.

Things the community contributed:
– Allow resolving of Enum values.
– Added Enum values to completion lists.
– Some tweaks to import optimizations.
– Added folding for documentation comments.

Next months goals (same as last month :/ ):
– Fix Flash debugging for non-OpenFL projects.
– Make a release.
– 2017.3 support and bugs fixed.
– More work toward parsing and syncing project files, leading to the automatic managing of compiler conditional definitions.

Have a great month, everybody!!

This month’s status report is a bit late coming because I wanted to report on some things I thought you’d be excited about!

Pre-release Build Available

There’s a test build available on my personal github fork.  You can read the release notes here.  A couple of highlights.

• Flash debugging fixes:  For OpenFL, We no longer use the “lime test” build command.  Instead we use its component “lime update,” “lime build,” and “lime run” commands. There should be much less chance of a time-out now that the debugger won’t start until after a successful build and won’t have to wait for the build to complete after the debugger has already started and is waiting for a connection.  In other words, the old method was: 1) start debugger (which waits for the app to connect) 2) lime test (which internally ran a build);  the new method is 1) lime update, 2) lime build, 3) start debugger, 4) lime run.  This also allows removing the Pre-Run “Build” task to be effective now — you can run the .SWF file from the last successful build.
• Some fixes to improve error recognition and reporting.  Flash target builds should now always stop if there is a build error.
• Folding has been implemented. You can now fold many types of sections within your code:
  + Imports and using sections (Thanks to Ilya Malanin! a.k.a. @mayakwd)
  + Classes, Methods, and, really, anything with curly braces (Also, thanks to Ilya.)
  + Compiler conditional sections (#if, etc.)
  + And, per FlowPlay’s specific request… Commented regions, so your sections marked with “//region Title … //endregion” can now be collapsed to a single line.
• Support for HL and CPPIA targets.

The test build also has a number of the new things that we expect to release in the next week or so.  We are waiting for me to get the IDEA 2017.3-specific bugs fixed and then it will roll out.  In the mean time, if you have some time, Please, Please, test it!

Other things I worked on this month:

• Removing IDEA v14 and 15 support.
• Adding IDEA 2018.1 build support.
• Parsing Haxe project files and keeping the IDEA project settings in sync with them. (Incomplete.)

Things the community contributed:

• Several NPE fixes for scenarios that I don’t usually see.  (Multiple contributors.)
• Initial folding support for import statements and body blocks. (Ilya)
• Wild-card imports optimization. (Ilya)
• Rework resolving for imports. (Ilya)

Next month’s goals:

• Flash target builds should now always stop if there is a build error.
• 0.12.0 release (or maybe we’ll call it 1.0?).
• More work toward parsing and syncing project files, leading to the automatic managing of compiler conditional definitions.