README
¶
Java Automatic Dependencies (Jadep)
Jadep is a Bazel BUILD file generator for Java projects. It adds BUILD
dependencies that a Java file needs, aiming for <1s response times.
Jadep is intended to manage BUILD files for your own code in the current Bazel workspace (as opposed to BUILD files for third-party libraries).
Jadep is not an official Google product.
Contents
Usage
~/bin/jadep path/to/File.java
Detailed Example: Migrating a Java project to Bazel
https://github.com/cgrushko/text/blob/master/migrating-gjf-to-bazel.md
Building / Installation
The following will build Jadep and its persistent server, and will copy them to
~/bin/ and ~/jadep/.
# Jadep
mkdir -p ~/bin
mkdir -p ~/jadep
bazel build -c opt //cmd/jadep
jadep=( bazel-bin/cmd/jadep/*/jadep ) # work around https://github.com/bazelbuild/rules_go/issues/1239
cp "${jadep[0]}" ~/bin/
# PackageLoader server
bazel build -c opt --nocheck_visibility //java/com/google/devtools/javatools/jade/pkgloader:GrpcLocalServer_deploy.jar
cp bazel-bin/java/com/google/devtools/javatools/jade/pkgloader/GrpcLocalServer_deploy.jar ~/jadep/
cp scripts/pkgloader_server.sh ~/jadep/
# JDK symbols [Jadep can run without these]
bazel build //:jdk_android_builtin_class_names
cp bazel-genfiles/jdk_android_builtin_class_names.txt ~/jadep/
How does it Work?
After parsing a Java file, Jadep extracts the class names it references.
It then tries to resolve each class name to BUILD rules that provide it, by
employing a set of strategies ("resolvers") in sequence.
Once a set of possible BUILD rules is found, it is filtered down according to
visibility, tags and so on.
The following subsections detail different parts of Jadep.
Detailed Flow
-
Connect to the PackageLoader server (GrpcLocalServer)
-
Jadep parses Java files to learn which fully-qualified names (FQNs) are referenced. This requires knowing which classes are defined in the same file (e.g., another inner class or a template type name) which is done by computing "jump-to-definition" information and then discarding all class names not defined in the same file.
Implemented in https://github.com/bazelbuild/tools_jvm_autodeps/blob/master/lang/java/parser/parser.go
-
The FQNs are passed to a sequence of "resolvers". A "resolver" returns BUILD rule candidates that can be used to satisfy a dependency on an FQN. Once a resolver returns a candidate for an FQN (i.e., it resolves it), the FQN is not passed on to additional resolvers. This is done to (a) improve performance and (b) allow ordering resolvers by accuracy to improve its quality.
The resolver interface is defined in https://github.com/bazelbuild/tools_jvm_autodeps/blob/2d9ab49baf4b1866abe0b4d670dd356ada30fbb4/jadeplib/jadeplib.go#L51
More details in the Resolver sections, below.
-
Candidates are filtered by visibility, tags, etc. Visibility sometimes requires interpreting multiple BUILD files, and care was taken to interpret as many as possible in parallel.
Code: https://github.com/bazelbuild/tools_jvm_autodeps/blob/master/filter/filter.go
-
Finally, Jadep asks the user which rule to add.
Extracting Class Names
Jadep parses a Java file to obtain an AST, then partially resolves it: each symbol is mapped to its place of definition. For example, a call to a method maps to the method's definition.
Jadep then walks the AST and finds all
- symbols that must be class names based on the Java 8 grammar
- symbols that can be class names, and aren't defined anywhere in the same Java file
Unqualified class names are assumed to be in the same package as the Java file.
This technique gives pretty good results, but the semantics of Java make it impossible to be 100% correct. For example, a subclass has access to all the (visible) inner classes of its superclass, without having to explicitly import them. Jadep doesn't follow inheritance chains because it means reading arbitrary files, so it doesn't know which symbols are inherited.
Resolver: File System
Java source files are typically organized in the file system according to their package and class name, and this resolver utilizes this structure to find BUILD rules.
It is based on the convention that a class named com.foo.Bar will be
defined in a file named <content root>/com/foo/Bar.java.
The <content root> is by default either one of {src/main/java, src/test/java}.
The resolver derives a set of file names from the set of content roots and a
transformation of the class names it's looking for, and searches for BUILD rules
that have these files in their srcs attributes.
The resolver also handles java_library.exports attributes and alias() rules
so long as they're in the same Bazel package as the composed file name.
Resolver: JDK / Android SDK
JDK class names (e.g. java.util.List) do not need any BUILD dependencies to
build, so this resolver simply maps these classes to nothing, ensuring that
Jadep won't add anything for them.
Bazel Android rules don't need dependencies for Android SDK classes, so this resolver also handles these classes.
Reading BUILD files
Since Jadep interacts with existing Bazel rules (e.g., when filtering by
visibility) it needs to read BUILD files.
We use Bazel's Skylark interpreter rather than Buildozer, because the latter is unable to interpret macros.
Since the Skylark interpreter is written in Java, a persistent local gRPC server is used to avoid repeatedly paying startup costs.
Extending / Hacking / Future Ideas
- The dictresolver.go is a resolver that uses a plain-text class -> BUILD mapping encoded in CSV, and can be used as an example for how to write a performant resolver.
- A Maven Central resolver would be useful - it would search class names in Maven Central and add their coordinates to a bazel-deps configuration.
- Kythe could be used to generate an index that Jadep uses.
Bugs
- Jadep doesn't yet handle external repositories. The
bazel.Labeldata structure is unaware of them, as isGrpcLocalServer.
Contributing
See CONTRIBUTING.md
Directories
¶
| Path | Synopsis |
|---|---|
|
Package bazel contains types representing Bazel concepts, such as packages and rules.
|
Package bazel contains types representing Bazel concepts, such as packages and rules. |
|
Package bazeldepsresolver resolves Java class names to precompiled jars set up by https://github.com/johnynek/bazel-deps/.
|
Package bazeldepsresolver resolves Java class names to precompiled jars set up by https://github.com/johnynek/bazel-deps/. |
|
Package buildozer provides functions to work with Buildozer
|
Package buildozer provides functions to work with Buildozer |
|
Package cli provides utilities for writing languages-specific main packages, such as Jade.
|
Package cli provides utilities for writing languages-specific main packages, such as Jade. |
|
cmd
|
|
|
jadep
The jadep command adds `BUILD` dependencies that a Java file needs.
|
The jadep command adds `BUILD` dependencies that a Java file needs. |
|
list_classes_in_jar
Program list_classes_in_jar consumes .jar files and prints a sorted list of Java classes they contains to stdout.
|
Program list_classes_in_jar consumes .jar files and prints a sorted list of Java classes they contains to stdout. |
|
Package color colorizes output sent to a terminal
|
Package color colorizes output sent to a terminal |
|
Package dictresolver resolves according to an in-memory map.
|
Package dictresolver resolves according to an in-memory map. |
|
Package filter provides functions to filter rules based on their use site.
|
Package filter provides functions to filter rules based on their use site. |
|
Package fsresolver resolves class names to Bazel rules using the file system.
|
Package fsresolver resolves class names to Bazel rules using the file system. |
|
Package future implements future/promise primitives.
|
Package future implements future/promise primitives. |
|
Package graphs provides functions related to graphs.
|
Package graphs provides functions related to graphs. |
|
Package grpcloader implements a Loader that connects to a gRPC server.
|
Package grpcloader implements a Loader that connects to a gRPC server. |
|
Package jadeplib finds a list of BUILD labels that provide the requested Java class names.
|
Package jadeplib finds a list of BUILD labels that provide the requested Java class names. |
|
Package jadepmain contains Jadep's main() function.
|
Package jadepmain contains Jadep's main() function. |
|
lang
|
|
|
java/parser
Package parser provides functions to parse Java code and reason about its contents.
|
Package parser provides functions to parse Java code and reason about its contents. |
|
java/parser/xrefs
Package xrefs provides support for local xrefs.
|
Package xrefs provides support for local xrefs. |
|
java/ruleconsts
Package ruleconsts defines constants related to names and kinds of Java rules.
|
Package ruleconsts defines constants related to names and kinds of Java rules. |
|
Package listclassesinjar lists class names in Jar files.
|
Package listclassesinjar lists class names in Jar files. |
|
Package loadertest provides test Loaders.
|
Package loadertest provides test Loaders. |
|
Package pkgloaderfakes provides fakes for structs returned from pkgloaderclient.
|
Package pkgloaderfakes provides fakes for structs returned from pkgloaderclient. |
|
Package pkgloading defines the package loading interface and implements functionality on top of it.
|
Package pkgloading defines the package loading interface and implements functionality on top of it. |
|
Package resolverutil provides functions that help implement resolvers.
|
Package resolverutil provides functions that help implement resolvers. |
|
Package sortingdepsranker ranks deps by simply sorting labels lexicographically.
|
Package sortingdepsranker ranks deps by simply sorting labels lexicographically. |
|
thirdparty
|
|
|
golang/parsers/ast
Package ast provides a basic tree implementation for storing an AST generated by a parser.
|
Package ast provides a basic tree implementation for storing an AST generated by a parser. |
|
golang/parsers/node
Package node provides types for working with AST nodes in a language-independent way.
|
Package node provides types for working with AST nodes in a language-independent way. |
|
golang/parsers/parsers
Package parsers is the main entry point for lexing and parsing source code for IDE and presentation purposes.
|
Package parsers is the main entry point for lexing and parsing source code for IDE and presentation purposes. |
|
golang/parsers/util/offset
Package offset provides a helper converting byte offsets in a string into (line, column) pairs, where 'column' represents the number of runes from the beginning of the line.
|
Package offset provides a helper converting byte offsets in a string into (line, column) pairs, where 'column' represents the number of runes from the beginning of the line. |
|
Package vlog implements conditional verbose/debug logging.
|
Package vlog implements conditional verbose/debug logging. |