Swarm MRUs: Adaptive frequency / Predictable lookups / API simplification (#17559)

* swarm/storage/mru: Adaptive Frequency

swarm/storage/mru/lookup: fixed getBaseTime
Added NewEpoch constructor

swarm/api/client: better error handling in GetResource()


swarm/storage/mru: Renamed structures.
Renamed ResourceMetadata to ResourceID. 
Renamed ResourceID.Name to ResourceID.Topic

swarm/storage/mru: Added binarySerializer interface and test tools

swarm/storage/mru/lookup: Changed base time to time and + marshallers

swarm/storage/mru:  Added ResourceID (former resourceMetadata)

swarm/storage/mru: Added ResourceViewId and serialization tests

swarm/storage/mru/lookup: fixed epoch unmarshaller. Added Epoch Equals

swarm/storage/mru: Fixes as per review comments

cmd/swarm: reworded resource create/update help text regarding topic

swarm/storage/mru: Added UpdateLookup and serializer tests

swarm/storage/mru: Added UpdateHeader, serializers and tests

swarm/storage/mru: changed UpdateAddr / epoch to Base()

swarm/storage/mru: Added resourceUpdate serializer and tests

swarm/storage/mru: Added SignedResourceUpdate tests and serializers

swarm/storage/mru/lookup: fixed GetFirstEpoch bug

swarm/storage/mru: refactor, comments, cleanup

Also added tests for Topic
swarm/storage/mru: handler tests pass

swarm/storage/mru: all resource package tests pass

swarm/storage/mru: resource test pass after adding
timestamp checking support

swarm/storage/mru: Added JSON serializers to ResourceIDView structures

swarm/storage/mru: Sever, client, API test pass

swarm/storage/mru: server test pass

swarm/storage/mru: Added topic length check

swarm/storage/mru: removed some literals,
improved "previous lookup" test case

swarm/storage/mru: some fixes and comments as per review

swarm/storage/mru: first working version without metadata chunk

swarm/storage/mru: Various fixes as per review

swarm/storage/mru: client test pass

swarm/storage/mru: resource query strings and manifest-less queries


swarm/storage/mru: simplify naming

swarm/storage/mru: first autofreq working version



swarm/storage/mru: renamed ToValues to AppendValues

swarm/resource/mru: Added ToValues / FromValues for URL query strings

swarm/storage/mru: Changed POST resource to work with query strings.
No more JSON.

swarm/storage/mru: removed resourceid

swarm/storage/mru: Opened up structures

swarm/storage/mru: Merged Request and SignedResourceUpdate

swarm/storage/mru: removed initial data from CLI resource create

swarm/storage/mru: Refactor Topic as a direct fixed-length array

swarm/storage/mru/lookup: Comprehensive GetNextLevel tests

swarm/storage/mru: Added comments

Added length checks in Topic
swarm/storage/mru: fixes in tests and some code comments

swarm/storage/mru/lookup: new optimized lookup algorithm

swarm/api: moved getResourceView to api out of server

swarm/storage/mru: Lookup algorithm working

swarm/storage/mru: comments and renamed NewLookupParams

Deleted commented code


swarm/storage/mru/lookup: renamed Epoch.LaterThan to After

swarm/storage/mru/lookup: Comments and tidying naming



swarm/storage/mru: fix lookup algorithm

swarm/storage/mru: exposed lookup hint
removed updateheader

swarm/storage/mru/lookup: changed GetNextEpoch for initial values

swarm/storage/mru: resource tests pass

swarm/storage/mru: valueSerializer interface and tests



swarm/storage/mru/lookup: Comments, improvements, fixes, more tests

swarm/storage/mru: renamed UpdateLookup to ID, LookupParams to Query

swarm/storage/mru: renamed query receiver var



swarm/cmd: MRU CLI tests

* cmd/swarm: remove rogue fmt

* swarm/storage/mru: Add version / header for future use

* swarm/storage/mru: Fixes/comments as per review

cmd/swarm: remove rogue fmt

swarm/storage/mru: Add version / header for future use-

* swarm/storage/mru: fix linter errors

* cmd/swarm: Speeded up TestCLIResourceUpdate

1 files changed, 43 insertions, 60 deletions

diff --git a/swarm/storage/mru/doc.go b/swarm/storage/mru/doc.go
index e1d7c2c34..19330e0c1 100644
--- a/swarm/storage/mru/doc.go
+++ b/swarm/storage/mru/doc.go
@@ -1,61 +1,44 @@
-// Package mru defines Mutable resource updates.
-// A Mutable Resource is an entity which allows updates to a resource
-// without resorting to ENS on each update.
-// The update scheme is built on swarm chunks with chunk keys following
-// a predictable, versionable pattern.
-//
-// Updates are defined to be periodic in nature, where the update frequency
-// is expressed in seconds.
-//
-// The root entry of a mutable resource is tied to a unique identifier that
-// is deterministically generated out of the metadata content that describes
-// the resource. This metadata includes a user-defined resource name, a resource
-// start time that indicates when the resource becomes valid,
-// the frequency in seconds with which the resource is expected to be updated, both of
-// which are stored as little-endian uint64 values in the database (for a
-// total of 16 bytes). It also contains the owner's address (ownerAddr)
-// This MRU info is stored in a separate content-addressed chunk
-// (call it the metadata chunk), with the following layout:
-//
-// (00|length|startTime|frequency|name|ownerAddr)
-//
-// (The two first zero-value bytes are used for disambiguation by the chunk validator,
-// and update chunk will always have a value > 0 there.)
-//
-// Each metadata chunk is identified by its rootAddr, calculated as follows:
-// metaHash=H(len(metadata), startTime, frequency,name)
-// rootAddr = H(metaHash, ownerAddr).
-// where H is the SHA3 hash function
-// This scheme effectively locks the root chunk so that only the owner of the private key
-// that ownerAddr was derived from can sign updates.
-//
-// The root entry tells the requester from when the mutable resource was
-// first added (Unix time in seconds) and in which moments to look for the
-// actual updates. Thus, a resource update for identifier "føø.bar"
-// starting at unix time 1528800000 with frequency 300 (every 5 mins) will have updates on 1528800300,
-// 1528800600, 1528800900 and so on.
-//
-// Actual data updates are also made in the form of swarm chunks. The keys
-// of the updates are the hash of a concatenation of properties as follows:
-//
-// updateAddr = H(period, version, rootAddr)
-// where H is the SHA3 hash function
-// The period is (currentTime - startTime) / frequency
-//
-// Using our previous example, this means that a period 3 will happen when the
-// clock hits 1528800900
-//
-// If more than one update is made in the same period, incremental
-// version numbers are used successively.
-//
-// A user looking up a resource would only need to know the rootAddr in order to get the versions
-//
-// the resource update data is:
-// resourcedata = headerlength|period|version|rootAddr|flags|metaHash
-// where flags is a 1-byte flags field. Flag 0 is set to 1 to indicate multihash
-//
-// the full update data that goes in the chunk payload is:
-// resourcedata|sign(resourcedata)
-//
-// headerlength is a 16 bit value containing the byte length of period|version|rootAddr|flags|metaHash
+/*
+Package mru defines Mutable resource updates.
+
+A Mutable Resource is an entity which allows updates to a resource
+without resorting to ENS on each update.
+The update scheme is built on swarm chunks with chunk keys following
+a predictable, versionable pattern.
+
+A Resource is tied to a unique identifier that is deterministically generated out of
+the chosen topic.
+
+A Resource View is defined as a specific user's point of view about a particular resource.
+Thus, a View is a Topic + the user's address (userAddr)
+
+Actual data updates are also made in the form of swarm chunks. The keys
+of the updates are the hash of a concatenation of properties as follows:
+
+updateAddr = H(View, Epoch ID)
+where H is the SHA3 hash function
+View is the combination of Topic and the user address
+Epoch ID is a time slot. See the lookup package for more information.
+
+A user looking up a resource would only need to know the View in order to
+another user's updates
+
+The resource update data is:
+resourcedata = View|Epoch|data
+
+the full update data that goes in the chunk payload is:
+resourcedata|sign(resourcedata)
+
+Structure Summary:
+
+Request: Resource update with signature
+	ResourceUpdate: headers + data
+		Header: Protocol version and reserved for future use placeholders
+		ID: Information about how to locate a specific update
+			View: Author of the update and what is updating
+				Topic: Item that the updates are about
+				User: User who updates the resource
+			Epoch: time slot where the update is stored
+
+*/
 package mru