diff options
author | Bob Summerwill <bob@summerwill.net> | 2016-08-01 13:25:37 +0800 |
---|---|---|
committer | Bob Summerwill <bob@summerwill.net> | 2016-08-01 16:45:11 +0800 |
commit | 4ee2114127f87b08b76b3ca94cde80a49cdc056a (patch) | |
tree | b680926d0da4aadfddae0db9567557802f2c2929 /scripts/travis-emscripten | |
parent | 56727d61a61e1485c8360f00700d766632ec7163 (diff) | |
download | dexon-solidity-4ee2114127f87b08b76b3ca94cde80a49cdc056a.tar dexon-solidity-4ee2114127f87b08b76b3ca94cde80a49cdc056a.tar.gz dexon-solidity-4ee2114127f87b08b76b3ca94cde80a49cdc056a.tar.bz2 dexon-solidity-4ee2114127f87b08b76b3ca94cde80a49cdc056a.tar.lz dexon-solidity-4ee2114127f87b08b76b3ca94cde80a49cdc056a.tar.xz dexon-solidity-4ee2114127f87b08b76b3ca94cde80a49cdc056a.tar.zst dexon-solidity-4ee2114127f87b08b76b3ca94cde80a49cdc056a.zip |
Make the Solidity repository standalone.
This commit is the culmination of several months of work to decouple Solidity from the webthree-umbrella so that it can be developed in parallel with cpp-ethereum (the Ethereum C++ runtime) and so that even for the Solidity unit-tests there is no hard-dependency onto the C++ runtime.
The Tests-over-IPC refactoring was a major step in the same process which was already committed.
This commit contains the following changes:
- A subset of the CMake functionality in webthree-helpers was extracted and tailored for Solidity into ./cmake. Further cleanup is certainly possible.
- A subset of the libdevcore functionality in libweb3core was extracted and tailored for Solidity into ./libdevcore. Further cleanup is certainly possible
- The gas price constants in EVMSchedule were orphaned into libevmasm.
- Some other refactorings and cleanups were made to sever unnecessary EVM dependencies in the Solidity unit-tests.
- TravisCI and Appveyor support was added, covering builds and running of the unit-tests (Linux and macOS only for now)
- A bug-fix was made to get the Tests-over-IPC running on macOS.
- There are still reliability issues in the unit-tests, which need immediate attention. The Travis build has been flipped to run the unit-tests 5 times, to try to flush these out.
- The Emscripten automation which was previously in webthree-umbrella was merged into the TravisCI automation here.
- The development ZIP deployment step has been commented out, but we will want to read that ONLY for release branch.
Further iteration on these changes will definitely be needed, but I feel these have got to sufficient maturity than holding them back further isn't winning us anything. It is go time :-)
Diffstat (limited to 'scripts/travis-emscripten')
-rwxr-xr-x | scripts/travis-emscripten/build_emscripten.sh | 134 | ||||
-rw-r--r-- | scripts/travis-emscripten/deploy_key.enc | bin | 0 -> 1680 bytes | |||
-rwxr-xr-x | scripts/travis-emscripten/install_deps.sh | 44 | ||||
-rwxr-xr-x | scripts/travis-emscripten/publish_binary.sh | 73 |
4 files changed, 251 insertions, 0 deletions
diff --git a/scripts/travis-emscripten/build_emscripten.sh b/scripts/travis-emscripten/build_emscripten.sh new file mode 100755 index 00000000..db409455 --- /dev/null +++ b/scripts/travis-emscripten/build_emscripten.sh @@ -0,0 +1,134 @@ +#!/usr/bin/env bash + +#------------------------------------------------------------------------------ +# This script builds the solidity binary using Emscripten. +# Emscripten is a way to compile C/C++ to JavaScript. +# +# http://kripken.github.io/emscripten-site/ +# +# First run install_dep.sh OUTSIDE of docker and then +# run this script inside a docker image trzeci/emscripten +# +# The documentation for solidity is hosted at: +# +# http://solidity.readthedocs.io/ +# +# ------------------------------------------------------------------------------ +# This file is part of solidity. +# +# solidity is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# solidity is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with solidity. If not, see <http://www.gnu.org/licenses/> +# +# (c) 2016 solidity contributors. +#------------------------------------------------------------------------------ + +set -ev + +# We need git for extracting the commit hash +apt-get update +apt-get -y install git-core + +export WORKSPACE=/src + +# CryptoPP +echo -en 'travis_fold:start:compiling_cryptopp\\r' +cd "$WORKSPACE/cryptopp" +# if .git exists, it is a fresh checkout, otherwise it comes from the cache +# and is already compiled +test -e .git && ( +emcmake cmake -DCRYPTOPP_LIBRARY_TYPE=STATIC -DCRYPTOPP_RUNTIME_TYPE=STATIC && emmake make -j 4 +ln -s . src/cryptopp || true +rm -rf .git +) +echo -en 'travis_fold:end:compiling_cryptopp\\r' + +# Json-CPP +echo -en 'travis_fold:start:compiling_jsoncpp\\r' +cd "$WORKSPACE/jsoncpp" +# if .git exists, it is a fresh checkout, otherwise it comes from the cache +# and is already compiled +test -e .git && ( +emcmake cmake -DJSONCPP_LIB_BUILD_STATIC=ON -DJSONCPP_LIB_BUILD_SHARED=OFF \ + -DJSONCPP_WITH_TESTS=OFF -DJSONCPP_WITH_POST_BUILD_UNITTEST=OFF \ + -G "Unix Makefiles" . +emmake make -j 4 +rm -rf .git +) +echo -en 'travis_fold:end:compiling_jsoncpp\\r' + +# Boost +echo -en 'travis_fold:start:compiling_boost\\r' +cd "$WORKSPACE"/boost_1_57_0 +# if b2 exists, it is a fresh checkout, otherwise it comes from the cache +# and is already compiled +test -e b2 && ( +sed -i 's|using gcc ;|using gcc : : /usr/local/bin/em++ ;|g' ./project-config.jam +sed -i 's|$(archiver\[1\])|/usr/local/bin/emar|g' ./tools/build/src/tools/gcc.jam +sed -i 's|$(ranlib\[1\])|/usr/local/bin/emranlib|g' ./tools/build/src/tools/gcc.jam +./b2 link=static variant=release threading=single runtime-link=static \ + thread system regex date_time chrono filesystem unit_test_framework program_options random +find . -name 'libboost*.a' -exec cp {} . \; +rm -rf b2 libs doc tools more bin.v2 status +) +echo -en 'travis_fold:end:compiling_boost\\r' + +# Build dependent components and solidity itself +echo -en 'travis_fold:start:compiling_solidity\\r' +cd $WORKSPACE +mkdir -p build +cd build +emcmake cmake \ + -DCMAKE_BUILD_TYPE=Release \ + -DEMSCRIPTEN=1 \ + -DCMAKE_CXX_COMPILER=em++ \ + -DCMAKE_C_COMPILER=emcc \ + -DBoost_FOUND=1 \ + -DBoost_USE_STATIC_LIBS=1 \ + -DBoost_USE_STATIC_RUNTIME=1 \ + -DBoost_INCLUDE_DIR="$WORKSPACE"/boost_1_57_0/ \ + -DBoost_CHRONO_LIBRARY="$WORKSPACE"/boost_1_57_0/libboost_chrono.a \ + -DBoost_CHRONO_LIBRARIES="$WORKSPACE"/boost_1_57_0/libboost_chrono.a \ + -DBoost_DATE_TIME_LIBRARY="$WORKSPACE"/boost_1_57_0/libboost_date_time.a \ + -DBoost_DATE_TIME_LIBRARIES="$WORKSPACE"/boost_1_57_0/libboost_date_time.a \ + -DBoost_FILESYSTEM_LIBRARY="$WORKSPACE"/boost_1_57_0/libboost_filesystem.a \ + -DBoost_FILESYSTEM_LIBRARIES="$WORKSPACE"/boost_1_57_0/libboost_filesystem.a \ + -DBoost_PROGRAM_OPTIONS_LIBRARY="$WORKSPACE"/boost_1_57_0/libboost_program_options.a \ + -DBoost_PROGRAM_OPTIONS_LIBRARIES="$WORKSPACE"/boost_1_57_0/libboost_program_options.a \ + -DBoost_RANDOM_LIBRARY="$WORKSPACE"/boost_1_57_0/libboost_random.a \ + -DBoost_RANDOM_LIBRARIES="$WORKSPACE"/boost_1_57_0/libboost_random.a \ + -DBoost_REGEX_LIBRARY="$WORKSPACE"/boost_1_57_0/libboost_regex.a \ + -DBoost_REGEX_LIBRARIES="$WORKSPACE"/boost_1_57_0/libboost_regex.a \ + -DBoost_SYSTEM_LIBRARY="$WORKSPACE"/boost_1_57_0/libboost_system.a \ + -DBoost_SYSTEM_LIBRARIES="$WORKSPACE"/boost_1_57_0/libboost_system.a \ + -DBoost_THREAD_LIBRARY="$WORKSPACE"/boost_1_57_0/libboost_thread.a \ + -DBoost_THREAD_LIBRARIES="$WORKSPACE"/boost_1_57_0/libboost_thread.a \ + -DBoost_UNIT_TEST_FRAMEWORK_LIBRARY="$WORKSPACE"/boost_1_57_0/libboost_unit_test_framework.a \ + -DBoost_UNIT_TEST_FRAMEWORK_LIBRARIES="$WORKSPACE"/boost_1_57_0/libboost_unit_test_framework.a \ + -DJSONCPP_LIBRARY="$WORKSPACE"/jsoncpp/src/lib_json/libjsoncpp.a \ + -DJSONCPP_INCLUDE_DIR="$WORKSPACE"/jsoncpp/include/ \ + -DCRYPTOPP_LIBRARY="$WORKSPACE"/cryptopp/src/libcryptlib.a \ + -DCRYPTOPP_INCLUDE_DIR="$WORKSPACE"/cryptopp/src/ \ + -DDev_DEVCORE_LIBRARY="$WORKSPACE"/solidity/build/libdevcore/libdevcore.a \ + -DEth_EVMASM_LIBRARY="$WORKSPACE"/solidity/build/libevmasm/libevmasm.a \ + -DETHASHCL=0 -DEVMJIT=0 -DETH_STATIC=1 -DSOLIDITY=1 -DFATDB=0 -DTESTS=0 -DTOOLS=0 \ + .. +emmake make -j 4 + +# TODO - This is a temporary solution to the permissions issue which we are seeing in TravisCI, +# where this Emscripten build generates files which the main build then cannot delete. +# Presumably different accounts being used? This needs wrapping in some conditional, so we +# can choose to build, or build-and-clean. +cd .. +rm -rf build + +echo -en 'travis_fold:end:compiling_solidity\\r' diff --git a/scripts/travis-emscripten/deploy_key.enc b/scripts/travis-emscripten/deploy_key.enc Binary files differnew file mode 100644 index 00000000..d613c32e --- /dev/null +++ b/scripts/travis-emscripten/deploy_key.enc diff --git a/scripts/travis-emscripten/install_deps.sh b/scripts/travis-emscripten/install_deps.sh new file mode 100755 index 00000000..2c0e9f26 --- /dev/null +++ b/scripts/travis-emscripten/install_deps.sh @@ -0,0 +1,44 @@ +#!/usr/bin/env bash + +#------------------------------------------------------------------------------ +# Bash script for installing pre-requisite packages for building solidity +# using Emscripten on Ubuntu Trusty. +# +# The documentation for solidity is hosted at: +# +# http://solidity.readthedocs.io/ +# +# ------------------------------------------------------------------------------ +# This file is part of solidity. +# +# solidity is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# solidity is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with solidity. If not, see <http://www.gnu.org/licenses/> +# +# (c) 2016 solidity contributors. +#------------------------------------------------------------------------------ + +set -ev + +echo -en 'travis_fold:start:installing_dependencies\\r' +test -e cryptopp -a -e cryptopp/src || git clone https://github.com/mmoss/cryptopp.git +test -e jsoncpp -a -e jsoncpp/include || git clone https://github.com/open-source-parsers/jsoncpp.git +test -e boost_1_57_0 -a -e boost_1_57_0/boost || ( +wget 'http://downloads.sourceforge.net/project/boost/boost/'\ +'1.57.0/boost_1_57_0.tar.bz2?r=http%3A%2F%2Fsourceforge.net%2F'\ +'projects%2Fboost%2Ffiles%2Fboost%2F1.57.0%2F&ts=1421887207'\ + -O - | tar xj +cd boost_1_57_0 +./bootstrap.sh --with-toolset=gcc --with-libraries=thread,system,regex,date_time,chrono,filesystem,program_options,random +) +cd .. +echo -en 'travis_fold:end:installing_dependencies\\r' diff --git a/scripts/travis-emscripten/publish_binary.sh b/scripts/travis-emscripten/publish_binary.sh new file mode 100755 index 00000000..f4139539 --- /dev/null +++ b/scripts/travis-emscripten/publish_binary.sh @@ -0,0 +1,73 @@ +#!/usr/bin/env bash + +#------------------------------------------------------------------------------ +# Bash script for publishing Solidity Emscripten binaries to Github. +# +# The results are committed to https://github.com/ethereum/solc-bin. +# +# The documentation for solidity is hosted at: +# +# http://solidity.readthedocs.io/ +# +# ------------------------------------------------------------------------------ +# This file is part of solidity. +# +# solidity is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# solidity is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with solidity. If not, see <http://www.gnu.org/licenses/> +# +# (c) 2016 solidity contributors. +#------------------------------------------------------------------------------ + +set -e + +cd solidity +VER=$(cat CMakeLists.txt | grep 'set(PROJECT_VERSION' | sed -e 's/.*set(PROJECT_VERSION "\(.*\)".*/\1/') +test -n "$VER" +VER="v$VER" +COMMIT=$(git rev-parse --short HEAD) +DATE=$(date --date="$(git log -1 --date=iso --format=%ad HEAD)" --utc +%F) +cp build/solc/soljson.js "../soljson-$VER-$DATE-$COMMIT.js" +cd .. + + +ENCRYPTED_KEY_VAR="encrypted_${ENCRYPTION_LABEL}_key" +ENCRYPTED_IV_VAR="encrypted_${ENCRYPTION_LABEL}_iv" +ENCRYPTED_KEY=${!ENCRYPTED_KEY_VAR} +ENCRYPTED_IV=${!ENCRYPTED_IV_VAR} +openssl aes-256-cbc -K $ENCRYPTED_KEY -iv $ENCRYPTED_IV -in scripts/travis-emscripten/deploy_key.enc -out deploy_key -d +chmod 600 deploy_key +eval `ssh-agent -s` +ssh-add deploy_key + +git clone --depth 2 git@github.com:ethereum/solc-bin.git +cd solc-bin +git config user.name "travis" +git config user.email "chris@ethereum.org" +git checkout -B gh-pages origin/gh-pages +git clean -f -d -x +# We only want one release per day and we do not want to push the same commit twice. +if ls ./bin/soljson-"$VER-$DATE"-*.js ./bin/soljson-*-"$COMMIT.js" > /dev/null +then + true +else + cp ../soljson-*.js ./bin/ + ./update-index.sh + cd bin + LATEST=$(ls -r soljson-v* | head -n 1) + cp "$LATEST" soljson-latest.js + cp soljson-latest.js ../soljson.js + git add . + git add ../soljson.js + git commit -m "Added compiler version $LATEST" + git push origin gh-pages +fi |