swarm/storage/mru: Client-side MRU signatures (#784)

* swarm/storage/mru: Add embedded publickey and remove ENS dep

This commit breaks swarm, swarm/api...
but tests in swarm/storage/mru pass

* swarm: Refactor swarm, swarm/api to mru changes, make tests pass

* swarm/storage/mru: Remove self from recv, remove test ens vldtr

* swarm/storage/mru: Remove redundant test, expose ResourceHash mthd

* swarm/storage/mru: Make HeaderGetter mandatory + godoc fixes

* swarm/storage: Remove validator prefix for metadata chunk

* swarm/storage/mru: Use Address instead of PublicKey

* swarm/storage/mru: Change index from name to metadata chunk addr

* swarm/storage/mru: Refactor swarm/api/... to MRU index changes

* swarm/storage/mru: Refactor cleanup

* swarm/storage/mru: Rebase cleanup

* swarm: Use constructor for GenericSigner MRU in swarm.go

* swarm/storage: Change to BMTHash for MRU hashing

* swarm/storage: Reduce loglevel on chunk validator logs

* swarm/storage/mru: Delint

* swarm: MRU Rebase cleanup

* swarm/storage/mru: client-side mru signatures

Rebase to PR #668 and fix all conflicts

* swarm/storage/mru:  refactor and documentation

* swarm/resource/mru: error-checking  tests for parseUpdate/newUpdateChunk

* swarm/storage/mru: Added resourcemetadata tests

* swarm/storage/mru: Added tests  for UpdateRequest

* swarm/storage/mru: more test coverage for UpdateRequest and comments

* swarm/storage/mru: Avoid fake chunks in parseUpdate()

* swarm/storage/mru: Documented resource.go extensively

moved some functions where they make most sense

* swarm/storage/mru: increase test coverage for UpdateRequest and

variable name changes throughout to increase consistency

* swarm/storage/mru: moved default timestamp to NewCreateRequest-

* swarm/storage/mru: lookup refactor

* swarm/storage/mru: added comments and renamed raw flag to rawmru

* swarm/storage/mru: fix receiver typo

* swarm/storage/mru: refactored update chunk new/create

* swarm/storage/mru:  refactored signature digest to avoid malleability

* swarm/storage/mru: optimize update data serialization

* swarm/storage/mru: refactor and cleanup

* swarm/storage/mru: add timestamp struct and serialization

* swarm/storage/mru: fix lint error and mark some old code for deletion

* swarm/storage/mru: remove unnecessary variable

* swarm/storage/mru: Added more comments throughout

* swarm/storage/mru: Refactored metadata chunk layout + extensive error...

* swarm/storage/mru: refactor cli parser
Changed resource info output to JSON

* swarm/storage/mru: refactor serialization for extensibility

refactored error messages to NewErrorf

* swarm/storage/mru: Moved Signature to resource_sign.
Check Sign errors in server tests

* swarm/storage/mru: Remove isSafeName() checks

* swarm/storage/mru: scrubbed off all references to "block" for time

* swarm/storage/mru: removed superfluous isSynced() call.

* swarm/storage/mru: remove isMultihash() and ToSafeName functions

* swarm/storage/mru: various fixes and comments

* swarm/storage/mru: decoupled cli for independent create/update
* Made resource name optional
* Removed unused LookupPrevious

* swarm/storage/mru: Decoupled resource create / update & refactor

* swarm/storage/mru: Fixed some comments as per issues raised in PR #743

* swarm/storage/mru: Cosmetic changes as per #743 comments

* swarm/storage/mru: refct request encoder/decoder > marshal/unmarshal

* swarm/storage/mru: Cosmetic changes as per review in #748

* swarm/storage/mru: removed timestamp proof placeholder

* swarm/storage/mru: cosmetic/doc/fixes changes as per comments in #704

* swarm/storage/mru: removed unnecessary check in Handler.update

* swarm/storage/mru: Implemented Marshaler/Unmarshaler iface in Request

* swarm/storage/mru: Fixed linter error

* swarm/storage/mru: removed redundant address in signature digest

* swarm/storage/mru: fixed bug: LookupLatestVersionInPeriod not working

* swarm/storage/mru: Unfold Request creation API for create or update+create
set common time source for mru package

* swarm/api/http: fix HandleGetResource error variable shadowed
when requesting a resource that does not exist

* swarm/storage/mru: Add simple check to detect duplicate updates

* swarm/storage/mru: moved Multihash() to the right place.

* cmd/swarm: remove unneeded clientaccountmanager.go

* swarm/storage/mru: Changed some comments as per reviews in #784

* swarm/storage/mru: Made SignedResourceUpdate.GetDigest() public

* swarm/storage/mru: cosmetic changes as per comments in #784

* cmd/swarm: Inverted --multihash flag default

* swarm/storage/mru: removed Verify from SignedResourceUpdate.fromChunk

* swarm/storage/mru: Moved validation code out of serializer
Cosmetic / comment changes

* swarm/storage/mru: Added unit tests for UpdateLookup

* swarm/storage/mru: Increased coverage of metadata serialization

* swarm/storage/mru: Increased test coverage of updateHeader serializers

* swarm/storage/mru: Add resourceUpdate serializer test

1 files changed, 61 insertions, 0 deletions

diff --git a/swarm/storage/mru/doc.go b/swarm/storage/mru/doc.go
new file mode 100644
index 000000000..e1d7c2c34
--- /dev/null
+++ b/swarm/storage/mru/doc.go
@@ -0,0 +1,61 @@
+// Package mru defines Mutable resource updates.
+// A Mutable Resource is an entity which allows updates to a resource
+// without resorting to ENS on each update.
+// The update scheme is built on swarm chunks with chunk keys following
+// a predictable, versionable pattern.
+//
+// Updates are defined to be periodic in nature, where the update frequency
+// is expressed in seconds.
+//
+// The root entry of a mutable resource is tied to a unique identifier that
+// is deterministically generated out of the metadata content that describes
+// the resource. This metadata includes a user-defined resource name, a resource
+// start time that indicates when the resource becomes valid,
+// the frequency in seconds with which the resource is expected to be updated, both of
+// which are stored as little-endian uint64 values in the database (for a
+// total of 16 bytes). It also contains the owner's address (ownerAddr)
+// This MRU info is stored in a separate content-addressed chunk
+// (call it the metadata chunk), with the following layout:
+//
+// (00|length|startTime|frequency|name|ownerAddr)
+//
+// (The two first zero-value bytes are used for disambiguation by the chunk validator,
+// and update chunk will always have a value > 0 there.)
+//
+// Each metadata chunk is identified by its rootAddr, calculated as follows:
+// metaHash=H(len(metadata), startTime, frequency,name)
+// rootAddr = H(metaHash, ownerAddr).
+// where H is the SHA3 hash function
+// This scheme effectively locks the root chunk so that only the owner of the private key
+// that ownerAddr was derived from can sign updates.
+//
+// The root entry tells the requester from when the mutable resource was
+// first added (Unix time in seconds) and in which moments to look for the
+// actual updates. Thus, a resource update for identifier "føø.bar"
+// starting at unix time 1528800000 with frequency 300 (every 5 mins) will have updates on 1528800300,
+// 1528800600, 1528800900 and so on.
+//
+// Actual data updates are also made in the form of swarm chunks. The keys
+// of the updates are the hash of a concatenation of properties as follows:
+//
+// updateAddr = H(period, version, rootAddr)
+// where H is the SHA3 hash function
+// The period is (currentTime - startTime) / frequency
+//
+// Using our previous example, this means that a period 3 will happen when the
+// clock hits 1528800900
+//
+// If more than one update is made in the same period, incremental
+// version numbers are used successively.
+//
+// A user looking up a resource would only need to know the rootAddr in order to get the versions
+//
+// the resource update data is:
+// resourcedata = headerlength|period|version|rootAddr|flags|metaHash
+// where flags is a 1-byte flags field. Flag 0 is set to 1 to indicate multihash
+//
+// the full update data that goes in the chunk payload is:
+// resourcedata|sign(resourcedata)
+//
+// headerlength is a 16 bit value containing the byte length of period|version|rootAddr|flags|metaHash
+package mru