Merge pull request #1387 from ethereum/json-interface

JSON interface description

3 files changed, 267 insertions, 39 deletions

diff --git a/docs/index.rst b/docs/index.rst
index cb79687b..904c3a54 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -130,6 +130,7 @@ Contents
    solidity-by-example.rst
    solidity-in-depth.rst
    security-considerations.rst
+   using-the-compiler.rst
    style-guide.rst
    common-patterns.rst
    contributing.rst
diff --git a/docs/miscellaneous.rst b/docs/miscellaneous.rst
index cc40d6a7..47650067 100644
--- a/docs/miscellaneous.rst
+++ b/docs/miscellaneous.rst
@@ -221,45 +221,6 @@ This means the following source mappings represent the same information:
 
 ``1:2:1;:9;2::2;;``
 
-
-.. index:: ! commandline compiler, compiler;commandline, ! solc, ! linker
-
-.. _commandline-compiler:
-
-******************************
-Using the Commandline Compiler
-******************************
-
-One of the build targets of the Solidity repository is ``solc``, the solidity commandline compiler.
-Using ``solc --help`` provides you with an explanation of all options. The compiler can produce various outputs, ranging from simple binaries and assembly over an abstract syntax tree (parse tree) to estimations of gas usage.
-If you only want to compile a single file, you run it as ``solc --bin sourceFile.sol`` and it will print the binary. Before you deploy your contract, activate the optimizer while compiling using ``solc --optimize --bin sourceFile.sol``. If you want to get some of the more advanced output variants of ``solc``, it is probably better to tell it to output everything to separate files using ``solc -o outputDirectory --bin --ast --asm sourceFile.sol``.
-
-The commandline compiler will automatically read imported files from the filesystem, but
-it is also possible to provide path redirects using ``context:prefix=path`` in the following way:
-
-::
-
-    solc github.com/ethereum/dapp-bin/=/usr/local/lib/dapp-bin/ =/usr/local/lib/fallback file.sol
-
-This essentially instructs the compiler to search for anything starting with
-``github.com/ethereum/dapp-bin/`` under ``/usr/local/lib/dapp-bin`` and if it does not
-find the file there, it will look at ``/usr/local/lib/fallback`` (the empty prefix
-always matches). ``solc`` will not read files from the filesystem that lie outside of
-the remapping targets and outside of the directories where explicitly specified source
-files reside, so things like ``import "/etc/passwd";`` only work if you add ``=/`` as a remapping.
-
-You can restrict remappings to only certain source files by prefixing a context.
-
-The section on :ref:`import` provides more details on remappings.
-
-If there are multiple matches due to remappings, the one with the longest common prefix is selected.
-
-If your contracts use :ref:`libraries <libraries>`, you will notice that the bytecode contains substrings of the form ``__LibraryName______``. You can use ``solc`` as a linker meaning that it will insert the library addresses for you at those points:
-
-Either add ``--libraries "Math:0x12345678901234567890 Heap:0xabcdef0123456"`` to your command to provide an address for each library or store the string in a file (one library per line) and run ``solc`` using ``--libraries fileName``.
-
-If ``solc`` is called with the option ``--link``, all input files are interpreted to be unlinked binaries (hex-encoded) in the ``__LibraryName____``-format given above and are linked in-place (if the input is read from stdin, it is written to stdout). All options except ``--libraries`` are ignored (including ``-o``) in this case.
-
 *****************
 Contract Metadata
 *****************
diff --git a/docs/using-the-compiler.rst b/docs/using-the-compiler.rst
new file mode 100644
index 00000000..08f18132
--- /dev/null
+++ b/docs/using-the-compiler.rst
@@ -0,0 +1,266 @@
+******************
+Using the compiler
+******************
+
+.. index:: ! commandline compiler, compiler;commandline, ! solc, ! linker
+
+.. _commandline-compiler:
+
+Using the Commandline Compiler
+******************************
+
+One of the build targets of the Solidity repository is ``solc``, the solidity commandline compiler.
+Using ``solc --help`` provides you with an explanation of all options. The compiler can produce various outputs, ranging from simple binaries and assembly over an abstract syntax tree (parse tree) to estimations of gas usage.
+If you only want to compile a single file, you run it as ``solc --bin sourceFile.sol`` and it will print the binary. Before you deploy your contract, activate the optimizer while compiling using ``solc --optimize --bin sourceFile.sol``. If you want to get some of the more advanced output variants of ``solc``, it is probably better to tell it to output everything to separate files using ``solc -o outputDirectory --bin --ast --asm sourceFile.sol``.
+
+The commandline compiler will automatically read imported files from the filesystem, but
+it is also possible to provide path redirects using ``prefix=path`` in the following way:
+
+::
+
+    solc github.com/ethereum/dapp-bin/=/usr/local/lib/dapp-bin/ =/usr/local/lib/fallback file.sol
+
+This essentially instructs the compiler to search for anything starting with
+``github.com/ethereum/dapp-bin/`` under ``/usr/local/lib/dapp-bin`` and if it does not
+find the file there, it will look at ``/usr/local/lib/fallback`` (the empty prefix
+always matches). ``solc`` will not read files from the filesystem that lie outside of
+the remapping targets and outside of the directories where explicitly specified source
+files reside, so things like ``import "/etc/passwd";`` only work if you add ``=/`` as a remapping.
+
+If there are multiple matches due to remappings, the one with the longest common prefix is selected.
+
+If your contracts use :ref:`libraries <libraries>`, you will notice that the bytecode contains substrings of the form ``__LibraryName______``. You can use ``solc`` as a linker meaning that it will insert the library addresses for you at those points:
+
+Either add ``--libraries "Math:0x12345678901234567890 Heap:0xabcdef0123456"`` to your command to provide an address for each library or store the string in a file (one library per line) and run ``solc`` using ``--libraries fileName``.
+
+If ``solc`` is called with the option ``--link``, all input files are interpreted to be unlinked binaries (hex-encoded) in the ``__LibraryName____``-format given above and are linked in-place (if the input is read from stdin, it is written to stdout). All options except ``--libraries`` are ignored (including ``-o``) in this case.
+
+.. _compiler-api:
+
+Compiler Input and Output JSON Description
+******************************************
+
+.. warning::
+
+    This JSON interface is not yet supported by the Solidity compiler, but will be released in a future version.
+
+These JSON formats are used by the compiler API as well as are available through ``solc``. These are subject to change,
+some fields are optional (as noted), but it is aimed at to only make backwards compatible changes.
+
+The compiler API expects a JSON formatted input and outputs the compilation result in a JSON formatted output.
+
+Comments are of course not permitted and used here only for explanatory purposes.
+
+Input Description
+-----------------
+
+.. code-block:: none
+
+    {
+      // Required: Source code language, such as "Solidity", "serpent", "lll", "assembly", etc.
+      language: "Solidity",
+      // Required
+      sources:
+      {
+        // The keys here are the "global" names of the source files,
+        // imports can use other files via remappings (see below).
+        "myFile.sol":
+        {
+          // Optional: keccak256 hash of the source file
+          // It is used to verify the retrieved content if imported via URLs.
+          "keccak256": "0x123...",
+          // Required (unless "content" is used, see below): URL(s) to the source file.
+          // URL(s) should be imported in this order and the result checked against the
+          // keccak256 hash (if available). If the hash doesn't match or none of the
+          // URL(s) result in success, an error should be raised.
+          "urls":
+          [
+            "bzzr://56ab...",
+            "ipfs://Qma...",
+            "file:///tmp/path/to/file.sol"
+          ]
+        },
+        "mortal":
+        {
+          // Optional: keccak256 hash of the source file
+          "keccak256": "0x234...",
+          // Required (unless "urls" is used): literal contents of the source file
+          "content": "contract mortal is owned { function kill() { if (msg.sender == owner) selfdestruct(owner); } }"
+        }
+      },
+      // Optional
+      settings:
+      {
+        // Optional: Sorted list of remappings
+        remappings: [ ":g/dir" ],
+        // Optional: Optimizer settings (enabled defaults to false)
+        optimizer: {
+          enabled: true,
+          runs: 500
+        },
+        // Metadata settings (optional)
+        metadata: {
+          // Use only literal content and not URLs (false by default)
+          useLiteralContent: true
+        },
+        // Addresses of the libraries. If not all libraries are given here, it can result in unlinked objects whose output data is different.
+        libraries: {
+          // The top level key is the the name of the source file where the library is used.
+          // If remappings are used, this source file should match the global path after remappings were applied.
+          // If this key is an empty string, that refers to a global level.
+          "myFile.sol": {
+            "MyLib": "0x123123..."
+          }
+        }
+        // The following can be used to select desired outputs.
+        // If this field is omitted, then the compiler loads and does type checking, but will not generate any outputs apart from errors.
+        // The first level key is the file name and the second is the contract name, where empty contract name refers to the file itself,
+        // while the star refers to all of the contracts.
+        //
+        // The available output types are as follows:
+        //   abi - ABI
+        //   ast - AST of all source files
+        //   why3 - Why3 translated output
+        //   devdoc - Developer documentation (natspec)
+        //   userdoc - User documentation (natspec)
+        //   metadata - Metadata
+        //   evm.ir - New assembly format before desugaring
+        //   evm.assembly - New assembly format after desugaring
+        //   evm.legacyAssemblyJSON - Old-style assembly format in JSON
+        //   evm.opcodes - Opcodes list
+        //   evm.methodIdentifiers - The list of function hashes
+        //   evm.gasEstimates - Function gas estimates
+        //   evm.bytecode - Bytecode
+        //   evm.deployedBytecode - Deployed bytecode
+        //   evm.sourceMap - Source mapping (useful for debugging)
+        //   ewasm.wast - eWASM S-expressions format (not supported atm)
+        //   ewasm.wasm - eWASM binary format (not supported atm)
+        outputSelection: {
+          // Enable the metadata and bytecode outputs of every single contract.
+          "*": {
+            "*": [ "metadata", "evm.bytecode" ]
+          },
+          // Enable the abi and opcodes output of MyContract defined in file def.
+          "def": {
+            "MyContract": [ "abi", "evm.opcodes" ]
+          },
+          // Enable the source map output of every single contract.
+          "*": {
+            "*": [ "evm.sourceMap" ]
+          },
+          // Enable the AST and Why3 output of every single file.
+          "*": {
+            "": [ "ast", "why3" ]
+          }
+        }
+      }
+    }
+
+
+Output Description
+------------------
+
+.. code-block:: none
+
+    {
+      // Optional: not present if no errors/warnings were encountered
+      errors: [
+        {
+          // Optional: Location within the source file.
+          sourceLocation: {
+            file: "sourceFile.sol",
+            start: 0,
+            end: 100
+          ],
+          // Mandatory: Error type, such as "TypeError", "InternalCompilerError", "Exception", etc
+          type: "TypeError",
+          // Mandatory: Component where the error originated, such as "general", "why3", "ewasm", etc.
+          component: "general",
+          // Mandatory ("error" or "warning")
+          severity: "error",
+          // Mandatory
+          message: "Invalid keyword"
+        }
+      ],
+      // This contains the file-level outputs. In can be limited/filtered by the outputSelection settings.
+      sources: {
+        "sourceFile.sol": {
+          // Identifier (used in source maps)
+          id: 1,
+          // The AST object
+          ast: {}
+        }
+      },
+      // This contains the contract-level outputs. It can be limited/filtered by the outputSelection settings.
+      contracts: {
+        "sourceFile.sol": {
+          // If the language used has no contract names, this field should equal to an empty string.
+          "ContractName": {
+            // The Ethereum Contract ABI. If empty, it is represented as an empty array.
+            // See https://github.com/ethereum/wiki/wiki/Ethereum-Contract-ABI
+            abi: [],
+            evm: {
+              // Intermediate representation (string)
+              ir: "",
+              // Assembly (string)
+              assembly: "",
+              // Old-style assembly (string)
+              legacyAssemblyJSON: [],
+              // Bytecode and related details.
+              bytecode: {
+                // The bytecode as a hex string.
+                object: "00fe",
+                // The source mapping as a string. See the source mapping definition.
+                sourceMap: "",
+                // If given, this is an unlinked object.
+                linkReferences: {
+                  "libraryFile.sol": {
+                    // Byte offsets into the bytecode. Linking replaces the 20 bytes located there.
+                    "Library1": [
+                      { start: 0, length: 20 },
+                      { start: 200, length: 20 }
+                    ]
+                  }
+                }
+              }
+              // The same layout as above.
+              deployedBytecode: { },
+              // Opcodes list (string)
+              opcodes: "",
+              // The list of function hashes
+              methodIdentifiers: {
+                "5c19a95c": "delegate(address)",
+              },
+              // Function gas estimates
+              gasEstimates: {
+                creation: {
+                  dataCost: 420000,
+                  // -1 means infinite (aka. unknown)
+                  executionCost: -1
+                },
+                external: {
+                  "delegate(address)": 25000
+                },
+                internal: {
+                  "heavyLifting()": -1
+                }
+              }
+            },
+            // See the Metadata Output documentation
+            metadata: {},
+            ewasm: {
+              // S-expressions format
+              wast: "",
+              // Binary format (hex string)
+              wasm: ""
+            },
+            // User documentation (natspec)
+            userdoc: {},
+            // Developer documentation (natspec)
+            devdoc: {}
+          }
+        }
+      },
+      // Why3 output (string)
+      why3: ""
+    }