From 9b46752cc274154faf3fd9af2ec9587e02eb250a Mon Sep 17 00:00:00 2001 From: chriseth Date: Wed, 23 Nov 2016 19:04:57 +0100 Subject: Documentation. --- docs/miscellaneous.rst | 119 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 119 insertions(+) (limited to 'docs') diff --git a/docs/miscellaneous.rst b/docs/miscellaneous.rst index 9f733979..a09f297a 100644 --- a/docs/miscellaneous.rst +++ b/docs/miscellaneous.rst @@ -232,6 +232,125 @@ Either add ``--libraries "Math:0x12345678901234567890 Heap:0xabcdef0123456"`` to If ``solc`` is called with the option ``--link``, all input files are interpreted to be unlinked binaries (hex-encoded) in the ``__LibraryName____``-format given above and are linked in-place (if the input is read from stdin, it is written to stdout). All options except ``--libraries`` are ignored (including ``-o``) in this case. +***************** +Contract Metadata +***************** + +The Solidity compiler automatically generates an internal json file, the +contract metadata, that contains information about the current contract. +It can be used to query the compiler version, the sourcecode, the ABI +and NatSpec documentation in order to more safely interact with the contract +and to verify its source code. + +The compiler inserts a swarm hash of that file into the bytecode of each +contract, so that you can retrieve the file in an authenticated way +without having to resort to a centralized data provider. + +Specifically, the runtime code for a contract always starts with +``push32 pop``, so you can take a look at the 32 bytes starting at +the second byte of the code of a contract. + +Of course, you have to publish the metadata file to swarm (or some other service) +so that others can access it. The file can be output by using ``solc --metadata``. +It will contain swarm references to the source code, so you have to upload +all source files and the metadata. + +The metadata file has the following format. The example below is presented in a +human-readable way. Properly formatted metadata should use quotes correctly, +reduce whitespace to a minimum and sort the keys of all objects to arrive at a +unique formatting. +Comments are of course also not permitted and used here only for explanatory purposes. + + { + // Required: The version of the metadata format + version: "1", + // Required: Source code language, basically selects a "sub-version" + // of the specification + language: "Solidity", + // Required: Details about the compiler, contents are specific + // to the language. + compiler: { + // Required for Solidity: Version of the compiler + version: "0.4.6+commit.2dabbdf0.Emscripten.clang", + // Optional: Hash of the compiler binary which produced this output + keccak256: "0x123..." + }, + // Required: Compilation source files/source units, keys are file names + sources: + { + "myFile.sol": { + // Required: keccak256 hash of the source file + "keccak256": "0x123...", + // Required (unless "content" is used, see below): URL to the + // source file, protocol is more or less arbitrary, but a swarm + // URL is recommended + "url": "bzzr://56ab..." + }, + "mortal": { + "keccak256": "0x123...", + // Required (unless "url" is used): literal contents of the source file + "content": "contract mortal is owned { function kill() { if (msg.sender == owner) selfdestruct(owner); } }" + } + }, + // Required: Compiler settings + settings: + { + // Required for Solidity: Sorted list of remappings + remappings: [":g/dir"], + // Optional: Optimizer settings (enabled defaults to false) + optimizer: {enabled: true, runs: 500}, + // Required for Solidity: File and name of the contract or library this + // metadata is created for. + compilationTarget: { + "myFile.sol": "MyContract" + }, + // Required for Solidity: Addresses for libraries + libraries: { + "MyLib": "0x123123..." + } + }, + // Required: Generated information about the contract. + output: + { + // Required: ABI definition of the contract + abi: [ ... ], + // Required: NatSpec user documentation of the contract + userdoc: [ ... ], + // Required: NatSpec developer documentation of the contract + devdoc: [ ... ], + } + } + +Usage for Automatic Interface Generation and NatSpec +---------------------------------------------------- + +The metadata is used in the following way: A component that wants to interact +with a contract (e.g. mist) retrieves the code of the contract +and from that the first 33 bytes. If the first byte decodes into a PUSH32 +instruction, the other 32 bytes are interpreted as the swarm hash of +a file which is then retrieved. +That file is JSON-decoded into a structure like above. + +The component can then use the abi to automatically generate a rudimentary +user interface for the contract. + +Furthermore, mist can use the userdoc to display a confirmation message to the user +whenever they interact with the contract. + +Usage for Source Code Verification +---------------------------------- + +In order to verify the compilation, sources can be retrieved from swarm +via the link in the metadata file. +The compiler of the correct version (which is checked to be part of the "official" compilers) +is invoked on that input with the specified settings. The resulting +bytecode is compared to the data of the creation transaction or CREATE opcode data. +This automatically verifies the metadata since +its hash is part of the bytecode. +Excess data is constructor input data which should be decoded +according to the interface and presented to the user. + + *************** Tips and Tricks *************** -- cgit v1.2.3 From 91ecc4533dffbe67fa27adfaff27780ddf69c21a Mon Sep 17 00:00:00 2001 From: chriseth Date: Thu, 24 Nov 2016 10:32:52 +0100 Subject: Add swarm hash to the end of the bytecode. --- docs/miscellaneous.rst | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) (limited to 'docs') diff --git a/docs/miscellaneous.rst b/docs/miscellaneous.rst index a09f297a..1f94c387 100644 --- a/docs/miscellaneous.rst +++ b/docs/miscellaneous.rst @@ -242,14 +242,10 @@ It can be used to query the compiler version, the sourcecode, the ABI and NatSpec documentation in order to more safely interact with the contract and to verify its source code. -The compiler inserts a swarm hash of that file into the bytecode of each +The compiler appends a swarm hash (32 bytes) of that file to the end of the bytecode of each contract, so that you can retrieve the file in an authenticated way without having to resort to a centralized data provider. -Specifically, the runtime code for a contract always starts with -``push32 pop``, so you can take a look at the 32 bytes starting at -the second byte of the code of a contract. - Of course, you have to publish the metadata file to swarm (or some other service) so that others can access it. The file can be output by using ``solc --metadata``. It will contain swarm references to the source code, so you have to upload @@ -326,8 +322,7 @@ Usage for Automatic Interface Generation and NatSpec The metadata is used in the following way: A component that wants to interact with a contract (e.g. mist) retrieves the code of the contract -and from that the first 33 bytes. If the first byte decodes into a PUSH32 -instruction, the other 32 bytes are interpreted as the swarm hash of +and from that the last 32 bytes, which are interpreted as the swarm hash of a file which is then retrieved. That file is JSON-decoded into a structure like above. -- cgit v1.2.3 From 1316c0c872f5b94a73456a8151188dfbef317fdc Mon Sep 17 00:00:00 2001 From: chriseth Date: Thu, 24 Nov 2016 21:42:57 +0100 Subject: Fix documentation error. --- docs/miscellaneous.rst | 2 ++ 1 file changed, 2 insertions(+) (limited to 'docs') diff --git a/docs/miscellaneous.rst b/docs/miscellaneous.rst index 1f94c387..3b8c5232 100644 --- a/docs/miscellaneous.rst +++ b/docs/miscellaneous.rst @@ -257,6 +257,8 @@ reduce whitespace to a minimum and sort the keys of all objects to arrive at a unique formatting. Comments are of course also not permitted and used here only for explanatory purposes. +.. code-block:: json + { // Required: The version of the metadata format version: "1", -- cgit v1.2.3 From 6a7ff039df15be59fbb71dc3dfaad09fb0b8961f Mon Sep 17 00:00:00 2001 From: chriseth Date: Tue, 29 Nov 2016 17:47:47 +0100 Subject: Use CBOR encoding. --- docs/miscellaneous.rst | 36 +++++++++++++++++++++++++++--------- 1 file changed, 27 insertions(+), 9 deletions(-) (limited to 'docs') diff --git a/docs/miscellaneous.rst b/docs/miscellaneous.rst index 3b8c5232..5b51c4a9 100644 --- a/docs/miscellaneous.rst +++ b/docs/miscellaneous.rst @@ -236,20 +236,22 @@ If ``solc`` is called with the option ``--link``, all input files are interprete Contract Metadata ***************** -The Solidity compiler automatically generates an internal json file, the +The Solidity compiler automatically generates a json file, the contract metadata, that contains information about the current contract. It can be used to query the compiler version, the sourcecode, the ABI and NatSpec documentation in order to more safely interact with the contract and to verify its source code. -The compiler appends a swarm hash (32 bytes) of that file to the end of the bytecode of each -contract, so that you can retrieve the file in an authenticated way -without having to resort to a centralized data provider. +The compiler appends a swarm hash (32 bytes) of that file to the end of the +bytecode (for details, see below) of each contract, so that you can retrieve +the file in an authenticated way without having to resort to a centralized +data provider. Of course, you have to publish the metadata file to swarm (or some other service) -so that others can access it. The file can be output by using ``solc --metadata``. +so that others can access it. The file can be output by using ``solc --metadata`` +and the file will be called ``ContractName_meta.json``. It will contain swarm references to the source code, so you have to upload -all source files and the metadata. +all source files and the metadata file. The metadata file has the following format. The example below is presented in a human-readable way. Properly formatted metadata should use quotes correctly, @@ -319,13 +321,29 @@ Comments are of course also not permitted and used here only for explanatory pur } } + +Encoding of the Metadata Hash in the Bytecode +--------------------------------------------- + +Because we might support other ways to retrieve the metadata file in the future, +the mapping ``{"bzzr0": }`` is stored +[CBOR](https://tools.ietf.org/html/rfc7049)-encoded. Since the beginning of that +encoding is not easy to find, its length is added in a two-byte big-endian +encoding. The current version of the Solidity compiler thus adds the following +to the end of the deployed bytecode:: + + 0xa1 0x65 'b' 'z' 'z' 'r' '0' 0x58 0x20 <32 bytes swarm hash> 0x00 0x29 + +So in order to retrieve the data, you can check the end of the deployed +bytecode to match that pattern and use the swarm hash to retrieve the +file. + Usage for Automatic Interface Generation and NatSpec ---------------------------------------------------- The metadata is used in the following way: A component that wants to interact -with a contract (e.g. mist) retrieves the code of the contract -and from that the last 32 bytes, which are interpreted as the swarm hash of -a file which is then retrieved. +with a contract (e.g. mist) retrieves the code of the contract, from that +the swarm hash of a file which is then retrieved. That file is JSON-decoded into a structure like above. The component can then use the abi to automatically generate a rudimentary -- cgit v1.2.3 From 35325ee7c3e1af558caf0a47c8611cdfde959bb3 Mon Sep 17 00:00:00 2001 From: Alex Beregszaszi Date: Wed, 30 Nov 2016 10:12:28 +0000 Subject: Update metadata documentation --- docs/miscellaneous.rst | 46 ++++++++++++++++++++++++---------------------- 1 file changed, 24 insertions(+), 22 deletions(-) (limited to 'docs') diff --git a/docs/miscellaneous.rst b/docs/miscellaneous.rst index 5b51c4a9..c44b3e9e 100644 --- a/docs/miscellaneous.rst +++ b/docs/miscellaneous.rst @@ -236,21 +236,21 @@ If ``solc`` is called with the option ``--link``, all input files are interprete Contract Metadata ***************** -The Solidity compiler automatically generates a json file, the +The Solidity compiler automatically generates a JSON file, the contract metadata, that contains information about the current contract. -It can be used to query the compiler version, the sourcecode, the ABI +It can be used to query the compiler version, the sources used, the ABI and NatSpec documentation in order to more safely interact with the contract and to verify its source code. -The compiler appends a swarm hash (32 bytes) of that file to the end of the +The compiler appends a Swarm hash of the metadata file to the end of the bytecode (for details, see below) of each contract, so that you can retrieve the file in an authenticated way without having to resort to a centralized data provider. -Of course, you have to publish the metadata file to swarm (or some other service) +Of course, you have to publish the metadata file to Swarm (or some other service) so that others can access it. The file can be output by using ``solc --metadata`` and the file will be called ``ContractName_meta.json``. -It will contain swarm references to the source code, so you have to upload +It will contain Swarm references to the source code, so you have to upload all source files and the metadata file. The metadata file has the following format. The example below is presented in a @@ -282,12 +282,13 @@ Comments are of course also not permitted and used here only for explanatory pur // Required: keccak256 hash of the source file "keccak256": "0x123...", // Required (unless "content" is used, see below): URL to the - // source file, protocol is more or less arbitrary, but a swarm + // source file, protocol is more or less arbitrary, but a Swarm // URL is recommended "url": "bzzr://56ab..." }, "mortal": { - "keccak256": "0x123...", + // Required: keccak256 hash of the source file + "keccak256": "0x234...", // Required (unless "url" is used): literal contents of the source file "content": "contract mortal is owned { function kill() { if (msg.sender == owner) selfdestruct(owner); } }" } @@ -296,15 +297,18 @@ Comments are of course also not permitted and used here only for explanatory pur settings: { // Required for Solidity: Sorted list of remappings - remappings: [":g/dir"], + remappings: [ ":g/dir" ], // Optional: Optimizer settings (enabled defaults to false) - optimizer: {enabled: true, runs: 500}, + optimizer: { + enabled: true, + runs: 500 + }, // Required for Solidity: File and name of the contract or library this // metadata is created for. compilationTarget: { "myFile.sol": "MyContract" }, - // Required for Solidity: Addresses for libraries + // Required for Solidity: Addresses for libraries used libraries: { "MyLib": "0x123123..." } @@ -326,7 +330,7 @@ Encoding of the Metadata Hash in the Bytecode --------------------------------------------- Because we might support other ways to retrieve the metadata file in the future, -the mapping ``{"bzzr0": }`` is stored +the mapping ``{"bzzr0": }`` is stored [CBOR](https://tools.ietf.org/html/rfc7049)-encoded. Since the beginning of that encoding is not easy to find, its length is added in a two-byte big-endian encoding. The current version of the Solidity compiler thus adds the following @@ -334,35 +338,33 @@ to the end of the deployed bytecode:: 0xa1 0x65 'b' 'z' 'z' 'r' '0' 0x58 0x20 <32 bytes swarm hash> 0x00 0x29 -So in order to retrieve the data, you can check the end of the deployed -bytecode to match that pattern and use the swarm hash to retrieve the -file. +So in order to retrieve the data, the end of the deployed bytecode can be checked +to match that pattern and use the Swarm hash to retrieve the file. Usage for Automatic Interface Generation and NatSpec ---------------------------------------------------- The metadata is used in the following way: A component that wants to interact -with a contract (e.g. mist) retrieves the code of the contract, from that -the swarm hash of a file which is then retrieved. +with a contract (e.g. Mist) retrieves the code of the contract, from that +the Swarm hash of a file which is then retrieved. That file is JSON-decoded into a structure like above. -The component can then use the abi to automatically generate a rudimentary +The component can then use the ABI to automatically generate a rudimentary user interface for the contract. -Furthermore, mist can use the userdoc to display a confirmation message to the user +Furthermore, Mist can use the userdoc to display a confirmation message to the user whenever they interact with the contract. Usage for Source Code Verification ---------------------------------- -In order to verify the compilation, sources can be retrieved from swarm +In order to verify the compilation, sources can be retrieved from Swarm via the link in the metadata file. The compiler of the correct version (which is checked to be part of the "official" compilers) is invoked on that input with the specified settings. The resulting bytecode is compared to the data of the creation transaction or CREATE opcode data. -This automatically verifies the metadata since -its hash is part of the bytecode. -Excess data is constructor input data which should be decoded +This automatically verifies the metadata since its hash is part of the bytecode. +Excess data corresponds to the constructor input data, which should be decoded according to the interface and presented to the user. -- cgit v1.2.3 From ea7a6520b283f5dfc47c05d8b8088608d2b5f4f9 Mon Sep 17 00:00:00 2001 From: Alex Beregszaszi Date: Wed, 30 Nov 2016 12:16:55 +0000 Subject: Fix documentation errors Subheadings must use consistent format across a file. The metadata with comments cannot be parsed as JSON. --- docs/miscellaneous.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) (limited to 'docs') diff --git a/docs/miscellaneous.rst b/docs/miscellaneous.rst index c44b3e9e..1754752d 100644 --- a/docs/miscellaneous.rst +++ b/docs/miscellaneous.rst @@ -259,7 +259,7 @@ reduce whitespace to a minimum and sort the keys of all objects to arrive at a unique formatting. Comments are of course also not permitted and used here only for explanatory purposes. -.. code-block:: json +.. code-block:: none { // Required: The version of the metadata format @@ -327,7 +327,7 @@ Comments are of course also not permitted and used here only for explanatory pur Encoding of the Metadata Hash in the Bytecode ---------------------------------------------- +============================================= Because we might support other ways to retrieve the metadata file in the future, the mapping ``{"bzzr0": }`` is stored @@ -342,7 +342,7 @@ So in order to retrieve the data, the end of the deployed bytecode can be checke to match that pattern and use the Swarm hash to retrieve the file. Usage for Automatic Interface Generation and NatSpec ----------------------------------------------------- +==================================================== The metadata is used in the following way: A component that wants to interact with a contract (e.g. Mist) retrieves the code of the contract, from that @@ -356,7 +356,7 @@ Furthermore, Mist can use the userdoc to display a confirmation message to the u whenever they interact with the contract. Usage for Source Code Verification ----------------------------------- +================================== In order to verify the compilation, sources can be retrieved from Swarm via the link in the metadata file. -- cgit v1.2.3