Building a Custom Compiler Plugin for Error Checks in NetBeans
Java codebases maintained by teams in Sydney, Melbourne and Brisbane grow quickly once a fintech or government project hits its second year. Without a guardrail that fits the team's own conventions, the usual javac warnings feel too generic and problems only surface during integration testing. A custom compiler plugin fills that gap by turning house rules into messages that appear next to the offending line in NetBeans, before the change ships.
The NetBeans Java infrastructure exposes hooks that let a module participate in the same pipeline that drives the built-in hints. By implementing a few interfaces from the source tree, you can run your own logic over every compilation unit, mark up the AST, and let the IDE render the diagnostics alongside stock compiler output. The rest of this walkthrough sketches a plugin for project-specific mistakes.
NetBeans Compiler Hooks at a Glance
The IDE uses the standard Java Compiler API under the bonnet, but it layers its own listener model on top so that features such as hints, navigation and refactoring can react to each tree being processed. A plugin that wants to add diagnostics implements CustomRuleQuery and registers a factory through the layer file. Each time javac compiles a file, NetBeans hands the plugin a CompilationInfo object that contains the parsed Trees instance, the source path and the current task context.
This design means your rule shares the same compile cycle as the rest of the project, which keeps editor squiggles in sync with a full build run from ant or maven. The trade-off is that heavy work happens on the compile thread, so anything you add has to be lean or risk slowing down a developer in Adelaide waiting for a clean rebuild after a refactor.
Creating the Plugin Module Skeleton
Start a fresh NBM project through the New Project wizard and pick NetBeans Module as the type. Add dependencies on org.netbeans.modules.java.compiler, org.netbeans.api.java.source and org.openide.util. Once Maven downloads the artefacts, register your rule factory in the META-INF/services folder so the IDE discovers it without manual installation. Layer.xml is also worth updating if you want the rule to appear under Tools → Options → Editor → Hints.
A useful habit picked up from Melbourne Java meetups is to keep the rule class small and put the AST visitors in a separate package. That way you can unit-test the visitor in plain JUnit before wiring it into the SPI, which speeds up turnaround when iterating on the rule's logic. Once the skeleton builds, install it into your IDE with mvn nbm:run-ide and confirm the module shows up under Installed.
Walking the AST to Detect Problems
Inside the visitor, extend TreePathScanner to override the visit methods that matter for your rule. For a rule that bans System.out.println calls inside production sources, override visitMethodInvocation, inspect the Trees instance and confirm the receiver symbol matches java.io.PrintStream. When the match fires, build a Diagnostic and push it onto the CompilationInfo's diagnostics list.
For a more realistic example, imagine a trading desk in Sydney that requires BigDecimal for any monetary arithmetic. The visitor can scan BinaryTree nodes, look at the symbol types of the operands, and flag any expression where a double participates in a calculation that also references a BigDecimal. Catching these at compile time saves the team from a long weekend reconciling ledger discrepancies.
Registering the Check with the Build Pipeline
The CustomRuleQuery implementation decides whether the rule applies to a given compilation context. Return true for the scopes where you want the check active, such as java.source.above level 1.7 or a custom source folder used for the Perth mining software team's legacy adapters. Inside the createCompilationUnitHandler method, return your visitor wrapped in a CompilationUnitTask.
NetBeans calls into the task once per file, so make sure your state resets between invocations. A common bug seen in Brisbane fintech plugins is leftover state from the previous file that triggers spurious diagnostics when the IDE recompiles after a clean. Keep an eye on the event bus and clear any caches in the finish method to avoid confusing the editor.
Running and Debugging the Plugin Locally
Once the plugin is installed, open any project that matches its scope and trigger a clean compile. Bad code should be underlined with a marker that matches the severity you assigned. The Output window lists diagnostics too, which is handy when you want to confirm the same findings appear in a headless build. When the rule misfires, attach the debugger to the run-ide instance and step through the visitor to see which TreePath branch matched.
Performance tuning is worth its own pass once the rule behaves. A slow visitor will be felt most when developers trigger Compile on Save in heavy modules. The earlier walkthrough on profiling CPU usage covers the workflow well, including how to read the sampler output and decide whether the visitor itself or the Trees API lookup is the hot spot.
Tuning for Speed and Memory
Long-running plugins can inflate the memory footprint of an Australian developer's laptop, especially on the 16 GB machines issued by banks. Keep the visitor stateless and avoid building intermediate collections for diagnostics you are about to discard. Where you must cache, hold onto CompilationInfo keys rather than raw AST nodes, because the IDE recycles trees between cycles.
Practical habits that pay off in day-to-day use
- Gate expensive checks behind a fast pre-filter so trivial files skip the deep walk.
- Reuse a single Trees handle per file rather than calling CompilationInfo.getTrees() repeatedly.
- Prefer Tree.Kind tests over getSymbol() lookups when you only need shape information.
- Mark diagnostics as ERROR only when the rule is rock solid; use WARNING for softer hints.
If you decide your team's needs would be better served by a lighter framework, the path covered in Micronaut in NetBeans shows how to swap the compilation pipeline for compile-time annotation processing instead.
Choosing Between Plugin Styles
Not every project needs the same integration. Some teams are happy with a simple annotation processor, others want full editor awareness. The trade-offs across the most common choices are summarised below.
| Approach | Editor awareness | Build-time only | Performance impact | Best fit |
|---|---|---|---|---|
| NetBeans CompilationUnitTask | High | Both | Medium | House-style enforcement with live feedback |
| Annotation processor (javac) | Low | Build only | Low | Library-level constraints shipped with code |
| LSP-style external linter | High | External | Variable | Cross-IDE consistency across mixed teams |
| IDE scripting (Groovy/JavaScript) | Medium | Build only | Low | Quick prototypes before a real SPI rule |
Checklist before settling on an approach
- The team needs squiggles in the editor, not just build output.
- A rule applies to many projects and should ship separately.
- The rule has to access Trees or JavacTask internals.
- Performance during Compile on Save must stay smooth.