fpp/compiler

= Compiler

== Requirements

. A Unix environment, such as Linux, Mac OS, or the Windows Subsystem for Linux.

. The https://www.scala-sbt.org[Simple Build Tool (sbt)] for Scala.

. Version 11 of the Java Development Kit (JDK).
If you have a different version of Java installed on your system, then
you can do the following:

.. Install version 11 of the JDK.
For example, in MacPorts you can run
+
[source,bash]
----
% sudo port install openjdk11-graalvm
----

.. Set the environment variable `FPP_JAVA_HOME` to the home directory associated
with the installation.
For example, on Mac OS:
+
[source,bash]
----
% export FPP_JAVA_HOME=/Library/Java/JavaVirtualMachines/openjdk11-graalvm/Contents/Home
----

== Installing

To install in the `bin` directory, run `./install`.
This step also runs all the Scala unit tests.
To install to a different directory, run `./install` _dir-name_.

*Setting the shell path:*
Once you have installed the FPP compiler to a directory `FPP_INSTALL_DIR`, you
should put `FPP_INSTALL_DIR` in your shell path. For example, in bash:

[source,bash]
----
% export FPP_INSTALL_DIR=[path-to-fpp-install-dir]
% export PATH=$PATH:$FPP_INSTALL_DIR
----

That way you can run the tools from anywhere.

*Available tools:* The available tools are as follows:

* `fpp-check`
* `fpp-depend`
* `fpp-filenames`
* `fpp-format`
* `fpp-from-xml`
* `fpp-locate-defs`
* `fpp-locate-uses`
* `fpp-syntax`
* `fpp-to-cpp`
* `fpp-to-dict`
* `fpp-to-layout`

See https://github.com/fprime-community/fpp/wiki/Tools for more information.

*Installing with custom sbt flags:*
For some installations, you may need to pass special flags
to `sbt`.
For example, when installing FPP inside a Docker image,
you may need to use the flags `--batch -Dsbt.server.forcestart=true`.

To do this, set the environment variable `FPP_SBT_FLAGS`.
For example:

[source,bash]
----
% export FPP_SBT_FLAGS='--batch -Dsbt.server.forcestart=true'
% ./install
----

*Installing with custom java flags:*
For some installations, you may need to pass special flags
to the `java` command.
To do this, set the environment variable `FPP_JAVA_FLAGS`.

== Testing

There are two kinds of tests: unit tests written in Scala and
unit tests that exercise the command-line tools.

. Run `./fpp-sbt test` to run all Scala unit tests.

. Run `./test` to run all command-line unit tests.
Do this after running `./install`.
If you update the build, rerun `./install` before running `./test`.

Some of the command-line unit tests run `scalac` (the Scala compiler).
To run these tests successfully, you will need Scala 2.13.1 or greater
installed and available on the command line.

== Development

. Start up the sbt shell:
+
[source,bash]
----
% ./fpp-sbt
sbt:fpp-compiler>
----

. In the shell, run `compile` to compile everything, `test` to run all
Scala unit tests,
and `assembly` to create jar files.
The `install` script runs these commands automatically.
It also moves the jar files to the install directory and puts executable
shell scripts there.

== Cleaning

To clean the repository, run `./clean` in this directory.
This will run `sbt clean` and delete generated build
and test artifacts.

== Building Native Binaries

*Running the release script:*
You can use the `release` script in this directory to build native
binaries of the FPP tools.
The script uses the `native-image` tool of GraalVM to convert JVM (Java
Virtual Machine) programs stored as Java Archive (JAR) files to
native binaries.

To use the `release` script, you must carry out the following steps:

. Install GraalVM.
For example, using MacPorts on Mac OS:
+
[source,bash]
----
% sudo port install openjdk11-graalvm
----

. Set the global environment variable `FPP_GRAALVM_JAVA_HOME` to the `Home` directory
associated with the GraalVM installation in step 1.
For example, on Mac OS:
+
[source,bash]
----
% export FPP_GRAALVM_JAVA_HOME=/Library/Java/JavaVirtualMachines/openjdk11-graalvm/Contents/Home
----

. Run `sudo $FPP_GRAALVM_JAVA_HOME/bin/gu install native-image`
*You must rerun this step every time you update or reinstall GraalVM (step 1).*
If you update or reinstall GraalVM, and you forget to re-install `native-image`,
then your GraalVM installation and your `native-image` installation will be out
of sync, and the native image build may not work.

. Run `./release` in this directory.
The script will build the native binary tools and install them at
`native-fpp-[platform name]`.

. The previous step installs the native binary tools into the `bin`
directory.
To install the standard version of the FPP tools, run `./install`.

*Custom native-image flags:*
For some installations, you may need to pass special flags
to `native-image`.
To do this, set the environment variable `FPP_NATIVE_IMAGE_FLAGS`.
For example, to set the temporary directory used by `native-image`, you can run

[source,bash]
----
% export FPP_NATIVE_IMAGE_FLAGS='-H:TempDirectory=/path/to/tmp/directory'
% ./release
----

*Running the tracing agent:*
Occasionally when developing FPP, you may need to re-run the tracing
agent to capture new runtime behavior.
To do that, use the following procedure:

. Ensure that `native-image` is installed and up-to-date,
as described above.

. Run `./install-trace` to install JVM versions of the FPP tools with
tracing enabled in the `bin` directory.

. Edit the file
`compiler/lib/src/main/resources/META-INF/native-image/reflect-config.json`
so that it contains a pair of square brackets `[]` and nothing else.
This will clear out any old trace information.

. Run `./test` to use the installed tools to run the unit tests.
This step will run the tracing agent and update the trace files.
All the tests should pass.
If the tests go haywire with failures, the likely cause
is an errant `.lock` file left behind by the tracing agent.
Run `find . -name .lock` in this directory to find the file;
then delete it and retry.

. Set the environment variable `FPRIME` to point to the root of
a local copy of the
https://github.com/nasa/fprime[F Prime repository] or
a fork of the repository.

. Run `./trace-fprime` to trace F Prime projects that run in CI.

. Commit the updated trace files to this repository.

. Run `./install` to reinstall the standard version
of the tools.

. If you wish, run `./release` to build the native
image files with the new trace files and run the unit tests.
All the tests should pass.
Alternatively you can let CI on `nasa/fpp` do this step.