Draft: Account Synchronization Over Remote Storage
Add a sync
module, which implements the draft spec "Multi-Client State Synchronization." When the user first creates their account, everything exists locally. To add devices, they will need to add remote storage. Options for remote storage include SFTP, Crust, Dropbox, etc. All devices must share the same cloud storage. Using that remote storage, implemented by external library users, sync
can write a transaction log that allows all other devices connected to that remote storage to stay in synchronization with each other. This module includes the following major components:
- RemoteStorage interface, which must be implemented against SFTP, Crust, Dropbox, etc.
- TransactionLog, a log that writes to the remote storage.
- KV, a key-value store wrapping https://git.xx.network/elixxir/ekv with options to store values optionally against in the TransactionLog.
- Collector, a thread manager that reads other transaction logs on RemoteStorage and applies updates against the KV.
Merge request reports
Activity
added 5 commits
Toggle commit listadded 14 commits
- d9ebf931 - Implement header
- 1e5a6b56 - Merge branch 'projects/AccountSync' of git.xx.network:elixxir/client into XX-4459/Header
- f7c15a7f - Write tests and fix header marshaler
- 9824a67c - Fix type in docstrin
- fb91e246 - Update sync/header_test.go
- b383ed86 - Update sync/header_test.go
- b3deb00e - Update sync/header_test.go
- d5044af9 - Add newline edge check test for sync.Header
- 43b295a1 - Merge branch 'XX-4459/Header' of git.xx.network:elixxir/client into XX-4459/Header
- 4a52cac4 - Add error check for Unmarshal
- 7ab0471f - Remove bad file
- b19de663 - Respond to comments
- e70af1c5 - Clean up sync.Header marshaller/unmarshaller"
- 526484ed - Merge branch 'XX-4459/Header' into 'projects/AccountSync'
Toggle commit listadded 37 commits
- b61fb509 - Implement sync.RemoteStore interface
- 9e066ae6 - Document sync.FilesystemRemoteStorage
- 39d04a09 - Add tests for sync.FileSystemRemoteStorage
- 75d1a5a5 - Merge branch 'projects/AccountSync' of git.xx.network:elixxir/client into XX-4464/RemoteStorage
- 18c77aac - Documentation for sync.FileSystemRemoteStorage
- 0219712c - Merge branch 'projects/AccountSync' of git.xx.network:elixxir/client into XX-4465/TransactionLog
- 3cfa3c82 - WIP: Implement transaction log
- 41cbf7d2 - Implement sorted inseration for Transaction log
- 6aae4127 - Improve documentation
- a59f3d6d - Write tests for LocalStore
- 1c5ed2b4 - Use sort package for TransactionLog.Insert
- eea09542 - WIP: Implement tests for TransactionLog
- e449511f - Fix bugs in TransactionLog
- 73b87fc4 - Finalize tests for TransactionLog
- af16f668 - Clear transactionLog's buffer after serialization
- 8453d44d - Document TransactionLog tests
- 13ff388b - Finalize tests for TransactionLog
- 11b63711 - Add GetLastWrite to sync.RemoteStore interface
- d077f16c - Merge branch 'XX-4464/RemoteStorage' of git.xx.network:elixxir/client into XX-4465/TransactionLog
- f51a506c - Improve error messages
- 92dbb79d - Fix TransactionLog tests
- 7358d52d - Fix file writing bug
- 668525ab - WIP: Respond to MR review
- f3a44375 - Fix tests
- 35732983 - Implement de/serialization of sync.Transaction
- 1e7867fd - Implement de/serialize for sync.Header
- 2410336c - Clean up tests for sync.Transaction
- 0f9e54d1 - Implement de/serialization for sync.TransactionLog
- 1d791e2f - Clean up sync crypto operations
- 356b60b5 - Implement NewOrLoad functionality for TransactionLog
- 8f541424 - Miscellaneous fixes and clean up
- ac8a52c3 - Fix lock handling for sync.TransactionLog
- 519bb277 - Implement insertion sort for TransactionLog.Append
- de5913a9 - Clean up code
- 9ee5a85f - Fix tests
- 8c8bc19d - Merge branch 'XX-4465/TransactionLog' into 'XX-4464/RemoteStorage'
- 8611ecc8 - Merge branch 'XX-4464/RemoteStorage' into 'projects/AccountSync'
Toggle commit listadded 27 commits
-
8611ecc8...250728fb - 26 commits from branch
release
- 10a3b4bc - Merge branch 'release' of git.xx.network:elixxir/client into projects/AccountSync
-
8611ecc8...250728fb - 26 commits from branch
added 11 commits
- 7b999883 - WIP: Implement remoteKv
- c37a03a2 - Improve testing for transaction log
- 86ca571b - Improve remoteKV implementation
- 645f608f - Implement tests for remoteKV
- 7d63fb3b - Clean up
- 6a93fa48 - WIP: Respond to comments
- 1786f163 - Resolve MR comments
- ceba70a9 - Tests & documentation
- 0c1b5a75 - WIP: Light refactor of RemoteKV
- 497e3f43 - Fix broken test
- 8c211d17 - Merge branch 'XX-4502/RemoteKv' into 'projects/AccountSync'
Toggle commit listadded 128 commits
-
8c211d17...3d22d9a8 - 127 commits from branch
release
- d36482d5 - Merge branch 'release' of git.xx.network:elixxir/client into projects/AccountSync
-
8c211d17...3d22d9a8 - 127 commits from branch
added 27 commits
- b85ffbfd - WIP: RemoteKV bindings
- 1160bef6 - WIP: Work on docstrings
- 75c8b152 - WIP: Implement Local & Remote storage in bindings
- b87e6ab7 - Finish up initial implemenation
- a33e9925 - Fix recursion bug
- beaca583 - WIP: Respond to MR
- aad86a3a - Fix comments
- 6dacc479 - Fix documentation and interface issues
- cec58923 - Fix doc issues
- 3df8fae5 - Fix signature for ekv local store
- 01c1e4c4 - Add UpsertCallbacks interface to sync/
- e7dfebb2 - Modify bidnings for sync/ changes
- 1d6e5a9d - Modify signature for sync.UpsertCallback
- d28276e0 - Clean up bindings
- 97084552 - Fix bindings issue
- 58a302fd - Test bindings fix
- 14fba2b8 - WIP: Get bindings working
- 3859fa8b - Add explanatory comment
- 897b1b95 - Fix comments in bindings
- 8e073810 - Add GetList functionality to LocalStore
- f97a2169 - Add bindings call for GetList
- 34cbea40 - Merge branch 'XX-4547/RemoteKvBindings' of git.xx.network:elixxir/client into...
- 4ef74c34 - Make doc changes
- 1df551c6 - Fix tensing of documentation
- 1bfc8027 - Merge branch 'projects/AccountSync' of git.xx.network:elixxir/client into XX-4547/RemoteKvBindings
- 0ea82092 - Fix bindings comment
- dda723c2 - Merge branch 'XX-4547/RemoteKvBindings' into 'projects/AccountSync'
Toggle commit listadded 26 commits
-
797259dc...1b360bb9 - 25 commits from branch
release
- 025bee1a - Merge remote-tracking branch 'origin/release' into projects/AccountSync
-
797259dc...1b360bb9 - 25 commits from branch
added 24 commits
- 025bee1a...d2a4ed19 - 14 earlier commits
- 5da9a585 - update to ekv v0.2.2
- 0290eeae - WIP on test for checking if keys are synchronizing properly. It is currently not working
- d2da4e8e - WIP Test improvements
- 2d4032e9 - Update to add a wait for the remote write
- 2481426b - force an error
- f9ac4016 - Fix broken test
- b6addb15 - Remove duplicate prefix code path
- 5a52b740 - fix tests and bindings so that they compile and pass
- 028ae7a6 - Respond to comments and add some docstrings
- e5e221fb - Merge branch 'hotfix/KvPrefixes' into 'projects/AccountSync'
Toggle commit listrequested review from @benger
assigned to @carback1
added 18 commits
- 79f308ab...b9ade93e - 8 earlier commits
- 50cf2af4 - Fix small errors in collector.go
- be0cfb42 - Finish up tests
- 60d2c51c - Respond to MR comments
- 97bd165b - Merge branch 'projects/AccountSync' into XX-4501/Collector
- 3be58940 - Make the collector build and complete merge
- 76fa7fbe - Revert "Make the collector build and complete merge"
- ab3a11fa - fix bindings
- 0f2f5a65 - Fix tests and complete merge
- 5785f202 - limit collector tests to desktop for now
- 89903728 - Merge branch 'XX-4501/Collector' into 'projects/AccountSync'
Toggle commit listadded 18 commits
- 89903728...0b7e95f4 - 8 earlier commits
- 4b53fa08 - Map functions for remote KV and remote VersionedKV, with a test on the...
- b2744919 - Use error objects and Wrapf for instanceID errors. Rename them to start with Err.
- b1010d8f - Redo LoadInstanceID logic so it's clearer
- dbaf584e - Merge branch 'XX-4594/instanceid' into XX-4595/maps2
- 5b408ebd - KV -> internalKV. As a stopgap, made an interface for the KV functions we do...
- 18e3e322 - when we timeout, dump thread stacks
- 93c8df77 - Merge branch 'XX-4595/maps2' into 'XX-4594/instanceid'
- 0ef584be - move the atomic for openWrite threads into the go routine call
- 9b1cd803 - Wait for writes to complete before completing loading test
- a740323d - Merge branch 'XX-4594/instanceid' into 'projects/AccountSync'
Toggle commit list