diff --git a/README.md b/README.md index 8c29768b63c9fd9e3b119e6832f1931bdbf71b53..62b1ed0de9a3a23929243db3299027af71793ee2 100644 --- a/README.md +++ b/README.md @@ -3,28 +3,30 @@ [](https://gitlab.com/elixxir/client/commits/master) [](https://gitlab.com/elixxir/client/commits/master) -The xx network client is a library and related command line tool -that facilitate making full-featured xx clients for all platforms. The -command line tool can be built for any platform supported by -golang. The libraries are built for iOS and Android using -[gomobile](https://godoc.org/golang.org/x/mobile/cmd/gomobile). + +The client is a library and related command-line tool +that facilitates making full-featured xx clients for all platforms. It interfaces with the cMix system, enabling access +to all xx network messaging features, including end-to-end encryption and metadata protection. This repository contains everything necessary to implement all of the -xx network messaging features. These include the end-to-end encryption -and metadata protection. It also contains features to extend the base +xx network messaging features. It also contains features to extend the base messaging protocols. +The command-line tool accompanying the client library can be built for any platform supported by +golang. The libraries are built for iOS and Android using +[gomobile](https://godoc.org/golang.org/x/mobile/cmd/gomobile). + For library writers, the client requires a writable folder to store data, functions for receiving and approving requests for creating secure end-to-end messaging channels, for discovering users, and for receiving different types of messages. Details for implementing these -features are in the Library Overview section below. +features are in the [Library Overview section](#library-overview) below. -The client is open source software released under the simplified BSD License. +The client is open-source software released under the simplified BSD License. ## Command Line Usage -The command line tool is intended for testing xx network functionality and not +The command-line tool is intended for testing xx network functionality and not for regular user use. These instructions assume that you have [Go 1.17.X installed](https://go.dev/doc/install), and GCC installed for Cgo (such as `build-essential` on Debian or Ubuntu). @@ -47,31 +49,27 @@ GOOS=windows GOARCH=386 CGO_ENABLED=0 go build -ldflags '-w -s' -o release/clien GOOS=darwin GOARCH=amd64 CGO_ENABLED=0 go build -ldflags '-w -s' -o release/client.darwin64 main.go ``` -To get an NDF from a network gateway and the permissioning server, use the `getndf` subcommand. The `getndf` subcommand allows command line users to poll the NDF from both a gateway and the permissioning server without any pre-established client connection. It requires an IP address, port, and ssl certificate. You can download an ssl cert with: -``` -openssl s_client -showcerts -connect permissioning.prod.cmix.rip:11420 < /dev/null 2>&1 | openssl x509 -outform PEM > certfile.pem -``` +#### Fetching an NDF + +All actions performed with the client require a current [NDF](https://xxdk-dev.xx.network/technical-glossary#network-definition-file-ndf). The NDF is downloadable from the command line or [via an access point](https://xxdk-dev.xx.network/quick-reference#func-downloadandverifysignedndfwithurl) in the Client API. -Example usage for Gateways: +Use the `getndf` command to fetch the NDF via the command line. `getndf` enables command-line users to poll the NDF from a network gateway without any pre-established client connection: ``` -$ go run main.go getndf --gwhost localhost:8440 --cert ~/integration/keys/cmix.rip.crt | jq . | head -{ - "Timestamp": "2021-01-29T01:19:49.227246827Z", - "Gateways": [ - { - "Id": "BRM+Iotl6ujIGhjRddZMBdauapS7Z6jL0FJGq7IkUdYB", - "Address": ":8440", - "Tls_certificate": "-----BEGIN CERTIFICATE-----\nMIIDbDCCAlSgAwIBAgIJAOUNtZneIYECMA0GCSqGSIb3DQEBBQUAMGgxCzAJBgNV\nBAYTAlVTMRMwEQYDVQQIDApDYWxpZm9ybmlhMRIwEAYDVQQHDAlDbGFyZW1vbnQx\nGzAZBgNVBAoMElByaXZhdGVncml0eSBDb3JwLjETMBEGA1UEAwwKKi5jbWl4LnJp\ncDAeFw0xOTAzMDUxODM1NDNaFw0yOTAzMDIxODM1NDNaMGgxCzAJBgNVBAYTAlVT\nMRMwEQYDVQQIDApDYWxpZm9ybmlhMRIwEAYDVQQHDAlDbGFyZW1vbnQxGzAZBgNV\nBAoMElByaXZhdGVncml0eSBDb3JwLjETMBEGA1UEAwwKKi5jbWl4LnJpcDCCASIw\nDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAPP0WyVkfZA/CEd2DgKpcudn0oDh\nDwsjmx8LBDWsUgQzyLrFiVigfUmUefknUH3dTJjmiJtGqLsayCnWdqWLHPJYvFfs\nWYW0IGF93UG/4N5UAWO4okC3CYgKSi4ekpfw2zgZq0gmbzTnXcHF9gfmQ7jJUKSE\ntJPSNzXq+PZeJTC9zJAb4Lj8QzH18rDM8DaL2y1ns0Y2Hu0edBFn/OqavBJKb/uA\nm3AEjqeOhC7EQUjVamWlTBPt40+B/6aFJX5BYm2JFkRsGBIyBVL46MvC02MgzTT9\nbJIJfwqmBaTruwemNgzGu7Jk03hqqS1TUEvSI6/x8bVoba3orcKkf9HsDjECAwEA\nAaMZMBcwFQYDVR0RBA4wDIIKKi5jbWl4LnJpcDANBgkqhkiG9w0BAQUFAAOCAQEA\nneUocN4AbcQAC1+b3To8u5UGdaGxhcGyZBlAoenRVdjXK3lTjsMdMWb4QctgNfIf\nU/zuUn2mxTmF/ekP0gCCgtleZr9+DYKU5hlXk8K10uKxGD6EvoiXZzlfeUuotgp2\nqvI3ysOm/hvCfyEkqhfHtbxjV7j7v7eQFPbvNaXbLa0yr4C4vMK/Z09Ui9JrZ/Z4\ncyIkxfC6/rOqAirSdIp09EGiw7GM8guHyggE4IiZrDslT8V3xIl985cbCxSxeW1R\ntgH4rdEXuVe9+31oJhmXOE9ux2jCop9tEJMgWg7HStrJ5plPbb+HmjoX3nBO04E5\n6m52PyzMNV+2N21IPppKwA==\n-----END CERTIFICATE-----\n" - }, - { - "Id": "JCBd9mAQb2BW8hc8H9avy1ubcjUAa7MHrPp0dBU/VqQB", +// Fetch NDF (example usage for Gateways, assumes you are running a gateway locally) +$ go run main.go getndf --gwhost localhost:8440 --cert certfile.pem | jq . >ndf.json ``` -Example usage for the Permissioning server: +You can also download an NDF directly for different environments by using the `--env` flag: + +```go +$ go run main.go getndf --env mainnet | jq . >ndf.json +// Or, run via the binary (assuming 64-bit Windows): +$ ./client.win64 getndf --env mainnet | jq . >ndf.json +``` +Sample content of `ndf.json`: ``` -$ go run main.go getndf --permhost localhost:18000 --cert ~/integration/keys/cmix.rip.crt | jq . | head { "Timestamp": "2021-01-29T01:19:49.227246827Z", "Gateways": [ @@ -82,61 +80,76 @@ $ go run main.go getndf --permhost localhost:18000 --cert ~/integration/keys/cmi }, { "Id": "JCBd9mAQb2BW8hc8H9avy1ubcjUAa7MHrPp0dBU/VqQB", + ..... ``` -Basic command line usage, sending unsafe, unencrypted messages to yourself: +#### Sending safe messages between 2 users -``` -client --password user-password --ndf ndf.json -l client.log -s session-directory --writeContact user-contact.json --unsafe -m \"Hello World, without E2E Encryption\" -``` - -* `--password` is the password used to encrypt and load the session. -* `--ndf` is the network definition file, downloadable from the xx network - website when available. -* `-l` the file to write logs (user messages are still printed to stdout) -* `--writeContact` Output the user's contact information to this file. -* `--unsafe` Send message without encryption (necessary whenever you have not - already established an e2e channel) -* `-m` The message to send +To send messages with end-to-end encryption, you must first establish a connection +or [authenticated channel](https://xxdk-dev.xx.network/technical-glossary#authenticated-channel) with the other user: -The client defaults to sending to itself when not supplied. +``` +# Get user contact jsons for each client +$ client --password user1-password --ndf ndf.json -l client1.log -s user1session --writeContact user1-contact.json --unsafe -m "Hi to me, without E2E Encryption" +$ client --password user2-password --ndf ndf.json -l client2.log -s user2session --writeContact user2-contact.json --unsafe -m "Hi to me, without E2E Encryption" -Sending unsafe messages between 2 users: +# Request authenticated channel from another client. Note that the receiving client +# is expected to confirm the request before any specified timeout (default 120s) +$ client --password password --ndf ndf.json -l client.log -s session-directory --destfile user2-contact.json --waitTimeout 360 --unsafe-channel-creation --send-auth-request +WARNING: unsafe channel creation enabled +Adding authenticated channel for: Qm40C5hRUm7uhp5aATVWhSL6Mt+Z4JVBQrsEDvMORh4D +Message received: +Sending to Qm40C5hRUm7uhp5aATVWhSL6Mt+Z4JVBQrsEDvMORh4D: +Received 1 -``` -# Get user contact jsons -client --password user1-password --ndf ndf.json -l client1.log -s user1session --writeContact user1-contact.json --unsafe -m "Hi" -client --password user2-password --ndf ndf.json -l client2.log -s user2session --writeContact user2-contact.json --unsafe -m "Hi" - -# Send messages to each other, run them in the background so they both receive -# each other's messages -client --password user1-password --ndf ndf.json -l client1.log -s user1session --destfile user2-contact.json --unsafe -m "Hi User 2, from User 1 without E2E Encryption" & -client --password user2-password --ndf ndf.json -l client2.log -s user2session --destfile user1-contact.json --unsafe -m "Hi User 1, from User 2 without E2E Encryption" & -``` +# Alternatively, to accept an authenticated channel request implicitly +# (should be within the timeout window of requesting client, or the request will need to be resent): +$ client --password "password" --ndf ndf.json -l client.log -s session-directory --destfile user2-contact.json" --unsafe-channel-creation --waitTimeout 200 +Authentication channel request from: o+QpswTmnsuZve/QRz0j0RYNWqjgx4R5pACfO00Pe0cD +Sending to o+QpswTmnsuZve/QRz0j0RYNWqjgx4R5pACfO00Pe0cD: +Message received: +Received 1 +# Send E2E Messages +$ client --password user1-password --ndf ndf.json -l client1.log -s user1session --destfile user2-contact.json -m "Hi User 2, from User 1 with E2E Encryption" +Sending to Qm40C5hRUm7uhp5aATVWhSL6Mt+Z4JVBQrsEDvMORh4D: Hi User 2, from User 1 with E2E Encryption +Timed out! +Received 0 + +$ client --password user2-password --ndf ndf.json -l client1.log -s user2session --destfile user1-contact.json -m "Hi User 1, from User 2 with E2E Encryption" +Sending to o+QpswTmnsuZve/QRz0j0RYNWqjgx4R5pACfO00Pe0cD: Hi User 1, from User 2 with E2E Encryption +Timed out! +Received 0 +``` + +* `--password`: The password used to encrypt and load the session. +* `--ndf`: The network definition file. +* `-l`: The file to write logs (user messages are still printed to stdout). +* `-s`: The storage directory for client session data. +* `--writeContact`: Output the user's contact information to this file. * `--destfile` is used to specify the recipient. You can also use `--destid b64:...` using the user's base64 id which is printed in the logs. +* `--unsafe`: Send message without encryption (necessary whenever you have not + already established an e2e channel). +* `--unsafe-channel-creation` Auto-create and auto-accept channel requests. +* `-m`: The message to send. -To send with end to end encryption, you must first establish a connection -with the other user: +Note that the client defaults to sending to itself when a destination is not supplied. +This is why we've used the `--unsafe` flag when creating the user contact jsons. +However, when sending between users it is dropped in exchange for `--unsafe-channel-creation`. -``` -# Get user contact jsons -client --password user1-password --ndf ndf.json -l client1.log -s user1session --writeContact user1-contact.json --unsafe -m "Hi" -client --password user2-password --ndf ndf.json -l client2.log -s user2session --writeContact user2-contact.json --unsafe -m "Hi" +For the authenticated channel creation to be considered "safe" the user should be prompted. You can do this +on the command line by explicitly accepting the channel creation +when sending a request with `--send-auth-request` and/or explicitly accepting a request with +`--accept-channel`: -# Send E2E Messages -client --password user1-password --ndf ndf.json -l client1.log -s user1session --destfile user1-contact.json --unsafe-channel-creation -m "Hi User 2, from User 1 with E2E Encryption" & -client --password user2-password --ndf ndf.json -l client2.log -s user2session --destfile user1-contact.json --unsafe-channel-creation -m "Hi User 1, from User 2 with E2E Encryption" & ``` - -Note that we have dropped the `--unsafe` in exchange for: -* `--unsafe-channel-creation` Auto-create and auto-accept channel requests. - -To be considered "safe" the user should be prompted. You can do this -with the command line by explicitly accepting the channel creation -when sending and/or explicitly accepting a request with -`--accept-channel`. +$ client --password user-password --ndf ndf.json -l client.log -s session-directory --destfile user-contact.json --accept-channel +Authentication channel request from: yYAztmoCoAH2VIr00zPxnj/ZRvdiDdURjdDWys0KYI4D +Sending to yYAztmoCoAH2VIr00zPxnj/ZRvdiDdURjdDWys0KYI4D: +Message received: +Received 1 +``` Full usage of client can be found with `client --help`: @@ -204,7 +217,7 @@ Flags: Use "client [command] --help" for more information about a command. ``` -Note that the client cannot be used on the betanet with precanned user ids. +**Note:** The client cannot be used on the betanet with precanned user ids. ## Library Overview @@ -219,175 +232,13 @@ platforms. Clients need to perform the same actions *in the same order* as shown in `cmd/root.go`. Specifically, certain handlers need to be registered and -set up before starting network threads (i.e., before StartNetworkFollowers --- #2 below) and you cannot perform certain actions until the network -connection reaches the "healthy" state. Below are relevant code listings for -how to do these actions. +set up before starting network threads. Additionally, you cannot perform certain actions until the network connection reaches a "healthy" state. -the ndf is the network definition file, downloadable from the xx network -website when available. +See [main.go](https://git.xx.network/elixxir/xxdk-examples/-/blob/sample-messaging-app/sample-messaging-app/main.go) for relevant code listings on when and how to perform these actions. +The [Getting Started](https://xxdk-dev.xx.network/getting-started) guide provides further detail. -1. Creating and/or Loading a client: -``` - //create a new client if none exist - if _, err := os.Stat(storeDir); os.IsNotExist(err) { - // Load NDF - ndfPath := viper.GetString("ndf") - ndfJSON, err := ioutil.ReadFile(ndfPath) - if err != nil { - jww.FATAL.Panicf(err.Error()) - } - err = api.NewClient(string(ndfJSON), storeDir, - []byte(pass), regCode) - } - - if err != nil { - jww.FATAL.Panicf("%+v", err) - } - } - - //load the client - client, err := api.Login(storeDir, []byte(pass)) - if err != nil { - jww.FATAL.Panicf("%+v", err) - } -``` -2. Set up registration, authorization request handlers -``` - user := client.GetUser() - - // Set up reception handler - swboard := client.GetSwitchboard() - recvCh := make(chan message.Receive, 10000) // Needs to be large - // Note the name below is arbitrary - listenerID := swboard.RegisterChannel("DefaultCLIReceiver", - switchboard.AnyUser(), message.Text, recvCh) - jww.INFO.Printf("Message ListenerID: %v", listenerID) - - // Set up auth request handler, which simply prints the - // user id of the requestor. - authMgr := client.GetAuthRegistrar() - authMgr.AddGeneralRequestCallback(printChanRequest) -... -func printChanRequest(requestor contact.Contact, message string) { - msg := fmt.Sprintf("Authentication channel request from: %s\n", - requestor.ID) - jww.INFO.Printf(msg) - fmt.Printf(msg) - msg = fmt.Sprintf("Authentication channel request message: %s\n", message) - jww.INFO.Printf(msg) - fmt.Printf(msg) - // Or you can auto confirm with: - // err := client.ConfirmAuthenticatedChannel( - // requestor) - -} -``` - -3. Start network threads and wait until network is healthy: -``` - err = client.StartNetworkFollower() - if err != nil { - jww.FATAL.Panicf("%+v", err) - } - - // Wait until connected or crash on timeout - connected := make(chan bool, 10) - client.GetHealth().AddChannel(connected) - waitUntilConnected(connected) -... -func waitUntilConnected(connected chan bool) { - waitTimeout := time.Duration(viper.GetUint("waitTimeout")) - timeoutTimer := time.NewTimer(waitTimeout * time.Second) - isConnected := false - //Wait until we connect or panic if we can't by a timeout - for !isConnected { - select { - case isConnected = <-connected: - jww.INFO.Printf("Network Status: %v\n", - isConnected) - break - case <-timeoutTimer.C: - jww.FATAL.Panic("timeout on connection") - } - } -} -``` - -4. Adding authenticated channels (if we haven't done it yet) -``` - if client.HasAuthenticatedChannel(recipientID) { - jww.INFO.Printf("Authenticated channel already in place for %s", - recipientID) - return - } - // Check if a channel exists for this recipientID - recipientContact, err := client.GetAuthenticatedChannelRequest( - recipientID) - if err == nil { - jww.INFO.Printf("Accepting existing channel request for %s", - recipientID) - err := client.ConfirmAuthenticatedChannel(recipientContact) - if err != nil { - jww.FATAL.Panicf("%+v", err) - } - return - } else { - recipientContact = recipient - } - - me := client.GetUser().GetContact() - jww.INFO.Printf("Requesting auth channel from: %s", - recipientID) - err := client.RequestAuthenticatedChannel(recipientContact, - me, msg) - if err != nil { - jww.FATAL.Panicf("%+v", err) - } -``` - -5. Sending E2E and Unsafe Messages -``` - msg := message.Send{ - Recipient: recipientID, - Payload: []byte(msgBody), - MessageType: message.Text, - } - paramsE2E := params.GetDefaultE2E() - paramsUnsafe := params.GetDefaultUnsafe() - - fmt.Printf("Sending to %s: %s\n", recipientID, msgBody) - var roundIDs []id.Round - if unsafe { - roundIDs, err = client.SendUnsafe(msg, - paramsUnsafe) - } else { - roundIDs, _, err = client.SendE2E(msg, - paramsE2E) - } - if err != nil { - jww.FATAL.Panicf("%+v", err) - } - jww.INFO.Printf("RoundIDs: %+v\n", roundIDs) -``` -The "RoundIDs" are the rounds in which your message parts were sent. After those -rounds have completed on the network, you can assume that the message has "sent" -successfully. See the client interface section for info on how to access round -state changes. - -6. Receiving Messages (assuming you set the receiver above in step 2) -``` - timeoutTimer := time.NewTimer(waitTimeout * time.Second) - select { - case <-timeoutTimer.C: - fmt.Println("Timed out!") - break - case m := <-recvCh: - fmt.Printf("Message received: %s\n", string( - m.Payload)) - break - } -``` +You can also visit the [API Quick Reference](https://xxdk-dev.xx.network/quick-reference) +for information on the types and functions exposed by the Client API. The main entry point for developing with the client is `api/client` (or `bindings/client`). We recommend using go doc to explore: @@ -492,4 +343,4 @@ parts of the roadmap that are intended for the client: * Efficiency improvements - mechanisms for message pickup and network tracking * will evolve to allow tradeoffs and options for use -We also are always looking at how to simplify and improve the library interface. +We are also always looking at simplifying and improving the library interface. diff --git a/api/client.go b/api/client.go index 6560fe67353d905c77fe072ff371b9e3fa4b4f37..9d2bfe9795f391cc8bb4e409ec6a1c21f8423aa2 100644 --- a/api/client.go +++ b/api/client.go @@ -169,15 +169,15 @@ func NewVanityClient(ndfJSON, storageDir string, password []byte, // NewClientFromBackup constructs a new Client from an encrypted backup. The backup // is decrypted using the backupPassphrase. On success a successful client creation, -//// the function will return a JSON encoded list of the E2E partners -//// contained in the backup. +// the function will return a JSON encoded list of the E2E partners +// contained in the backup and a json-encoded string containing parameters stored in the backup func NewClientFromBackup(ndfJSON, storageDir string, sessionPassword, - backupPassphrase []byte, backupFileContents []byte) ([]*id.ID, error) { + backupPassphrase []byte, backupFileContents []byte) ([]*id.ID, string, error) { backUp := &backup.Backup{} err := backUp.Decrypt(string(backupPassphrase), backupFileContents) if err != nil { - return nil, errors.WithMessage(err, "Failed to unmarshal decrypted client contents.") + return nil, "", errors.WithMessage(err, "Failed to unmarshal decrypted client contents.") } usr := user.NewUserFromBackup(backUp) @@ -185,7 +185,7 @@ func NewClientFromBackup(ndfJSON, storageDir string, sessionPassword, // Parse the NDF def, err := parseNDF(ndfJSON) if err != nil { - return nil, err + return nil, "", err } cmixGrp, e2eGrp := decodeGroups(def) @@ -206,10 +206,10 @@ func NewClientFromBackup(ndfJSON, storageDir string, sessionPassword, //move the registration state to indicate registered with registration on proto client err = storageSess.ForwardRegistrationStatus(storage.PermissioningComplete) if err != nil { - return nil, err + return nil, "", err } - return backUp.Contacts.Identities, nil + return backUp.Contacts.Identities, backUp.JSONParams, nil } // OpenClient session, but don't connect to the network or log in diff --git a/api/results.go b/api/results.go index 04e91e15e7c6e3407ecfab759dd9eb1f598d9885..eb0403dee92b93be7ad90cc3ab2b4cafff28e8f8 100644 --- a/api/results.go +++ b/api/results.go @@ -4,6 +4,7 @@ // Use of this source code is governed by a license that can be found in the // // LICENSE file // /////////////////////////////////////////////////////////////////////////////// + package api import ( @@ -18,7 +19,7 @@ import ( "gitlab.com/xx_network/primitives/id" ) -// Enum of possible round results to pass back +// RoundResult is the enum of possible round results to pass back type RoundResult uint const ( @@ -40,7 +41,7 @@ func (rr RoundResult) String() string { } } -// Callback interface which reports the requested rounds. +// RoundEventCallback interface which reports the requested rounds. // Designed such that the caller may decide how much detail they need. // allRoundsSucceeded: // Returns false if any rounds in the round map were unsuccessful. @@ -60,7 +61,7 @@ type historicalRoundsComm interface { GetHost(hostId *id.ID) (*connect.Host, bool) } -// Adjudicates on the rounds requested. Checks if they are +// GetRoundResults adjudicates on the rounds requested. Checks if they are // older rounds or in progress rounds. func (c *Client) GetRoundResults(roundList []id.Round, timeout time.Duration, roundCallback RoundEventCallback) error { @@ -168,7 +169,7 @@ func (c *Client) getRoundResults(roundList []id.Round, timeout time.Duration, roundsResults[roundId] = Failed allRoundsSucceeded = false } - return + continue } allRoundsSucceeded = false anyRoundTimedOut = true diff --git a/api/utils.go b/api/utils.go index 2486d4d66143bb7fe6b660a9d23da26e70828652..799e3abf5a2410d15a1299c6d682ef801fdd439c 100644 --- a/api/utils.go +++ b/api/utils.go @@ -20,9 +20,9 @@ const ( // Maximum input image size (in bytes) maxSize int64 = 12000000 // Desired number of pixels in output image - desiredSize = 640*480 + desiredSize = 640 * 480 // Desired number of pixels in output image for preview - desiredPreviewSize = 32*24 + desiredPreviewSize = 32 * 24 ) // CompressJpeg takes a JPEG image in byte format @@ -76,7 +76,6 @@ func CompressJpeg(imgBytes []byte) ([]byte, error) { return newImgBuf.Bytes(), nil } - // CompressJpeg takes a JPEG image in byte format // and compresses it based on desired output size func CompressJpegForPreview(imgBytes []byte) ([]byte, error) { diff --git a/api/version_vars.go b/api/version_vars.go index c96f6402a0c5c0e998a03e8dd5fe5c8609fbac7b..b681e79ff65784d85fc3b31c182c6f51e09c8697 100644 --- a/api/version_vars.go +++ b/api/version_vars.go @@ -1,10 +1,10 @@ // Code generated by go generate; DO NOT EDIT. // This file was generated by robots at -// 2022-02-23 12:55:40.971513 -0600 CST m=+0.038316967 +// 2022-03-10 11:46:32.709433 -0600 CST m=+0.047217210 package api -const GITVERSION = `80d6fb9f Merge branch 'hotfix/RenableHistorical' into 'release'` -const SEMVER = "4.0.0" +const GITVERSION = `0d927b59 Update deps` +const SEMVER = "4.1.0" const DEPENDENCIES = `module gitlab.com/elixxir/client go 1.17 @@ -19,11 +19,11 @@ require ( github.com/spf13/jwalterweatherman v1.1.0 github.com/spf13/viper v1.7.1 gitlab.com/elixxir/bloomfilter v0.0.0-20200930191214-10e9ac31b228 - gitlab.com/elixxir/comms v0.0.4-0.20220222221859-c12e29bde218 - gitlab.com/elixxir/crypto v0.0.7-0.20220222221347-95c7ae58da6b + gitlab.com/elixxir/comms v0.0.4-0.20220308183624-c2183e687a03 + gitlab.com/elixxir/crypto v0.0.7-0.20220309234716-1ba339865787 gitlab.com/elixxir/ekv v0.1.6 gitlab.com/elixxir/primitives v0.0.3-0.20220222212109-d412a6e46623 - gitlab.com/xx_network/comms v0.0.4-0.20220222212058-5a37737af57e + gitlab.com/xx_network/comms v0.0.4-0.20220223205228-7c4974139569 gitlab.com/xx_network/crypto v0.0.5-0.20220222212031-750f7e8a01f4 gitlab.com/xx_network/primitives v0.0.4-0.20220222211843-901fa4a2d72b golang.org/x/crypto v0.0.0-20220128200615-198e4374d7ed diff --git a/auth/callback.go b/auth/callback.go index a437d834ccc7240cdcc5ee18ee85528c6fc99d8a..70b9c2843406205198cdb439666741b193c52518 100644 --- a/auth/callback.go +++ b/auth/callback.go @@ -107,10 +107,19 @@ func (m *Manager) handleRequest(cmixMsg format.Message, jww.TRACE.Printf("handleRequest ECRPAYLOAD: %v", baseFmt.GetEcrPayload()) jww.TRACE.Printf("handleRequest MAC: %v", cmixMsg.GetMac()) + ecrPayload := baseFmt.GetEcrPayload() success, payload := cAuth.Decrypt(myHistoricalPrivKey, - partnerPubKey, baseFmt.GetEcrPayload(), + partnerPubKey, ecrPayload, cmixMsg.GetMac(), grp) + if !success { + jww.WARN.Printf("Attempting to decrypt old request packet...") + ecrPayload = append(ecrPayload, baseFmt.GetVersion()) + success, payload = cAuth.Decrypt(myHistoricalPrivKey, + partnerPubKey, ecrPayload, + cmixMsg.GetMac(), grp) + } + if !success { jww.WARN.Printf("Received auth request failed " + "its mac check") @@ -317,8 +326,8 @@ func (m *Manager) handleRequest(cmixMsg format.Message, //create the contact, note that no facts are sent in the payload c := contact.Contact{ - ID: partnerID, - DhPubKey: partnerPubKey, + ID: partnerID.DeepCopy(), + DhPubKey: partnerPubKey.DeepCopy(), OwnershipProof: copySlice(ecrFmt.ownership), Facts: facts, } @@ -360,7 +369,7 @@ func (m *Manager) handleRequest(cmixMsg format.Message, partnerID, rndNum) cbList := m.resetCallbacks.Get(c.ID) for _, cb := range cbList { - ccb := cb.(interfaces.ResetCallback) + ccb := cb.(interfaces.ResetNotificationCallback) go ccb(c) } } @@ -531,7 +540,7 @@ func handleBaseFormat(cmixMsg format.Message, grp *cyclic.Group) (baseFormat, baseFmt, err := unmarshalBaseFormat(cmixMsg.GetContents(), grp.GetP().ByteLen()) - if err != nil { + if err != nil && baseFmt == nil { return baseFormat{}, nil, errors.WithMessage(err, "Failed to"+ " unmarshal auth") } @@ -542,5 +551,5 @@ func handleBaseFormat(cmixMsg format.Message, grp *cyclic.Group) (baseFormat, } partnerPubKey := grp.NewIntFromBytes(baseFmt.pubkey) - return baseFmt, partnerPubKey, nil + return *baseFmt, partnerPubKey, nil } diff --git a/auth/confirm.go b/auth/confirm.go index ecc32ac304282ad123125b761916e3ea9af9e697..28a5292aec2ad83f888c35e1f19feb32a55acea7 100644 --- a/auth/confirm.go +++ b/auth/confirm.go @@ -35,7 +35,7 @@ func (m *Manager) ConfirmRequestAuth(partner contact.Contact) (id.Round, error) // Cannot confirm already established channels if _, err := m.storage.E2e().GetPartner(partner.ID); err == nil { - em := fmt.Sprintf("Cannot FonritmRequestAuth for %s, "+ + em := fmt.Sprintf("Cannot ConfirmRequestAuth for %s, "+ "channel already exists. Ignoring", partner.ID) jww.WARN.Print(em) m.net.GetEventManager().Report(5, "Auth", diff --git a/auth/fmt.go b/auth/fmt.go index f267945be7934705f7046b948efc409c42f51d6d..cf38aaed44967d8a825a27451783bf5318038544 100644 --- a/auth/fmt.go +++ b/auth/fmt.go @@ -65,18 +65,18 @@ func buildBaseFormat(data []byte, pubkeySize int) baseFormat { return f } -func unmarshalBaseFormat(b []byte, pubkeySize int) (baseFormat, error) { +func unmarshalBaseFormat(b []byte, pubkeySize int) (*baseFormat, error) { if len(b) < pubkeySize { - return baseFormat{}, errors.New("Received baseFormat too small") + return nil, errors.New("Received baseFormat too small") } bfmt := buildBaseFormat(b, pubkeySize) version := bfmt.GetVersion() if version != requestFmtVersion { - return baseFormat{}, errors.Errorf( + return &bfmt, errors.Errorf( "Unknown baseFormat version: %d", version) } - return bfmt, nil + return &bfmt, nil } func (f baseFormat) Marshal() []byte { diff --git a/auth/fmt_test.go b/auth/fmt_test.go index 64785b399abc261400a346fceb68bc4ee0fc59c0..30580ec1bef400eb05409d3033076e9f0b00b4cd 100644 --- a/auth/fmt_test.go +++ b/auth/fmt_test.go @@ -151,10 +151,10 @@ func TestBaseFormat_MarshalUnmarshal(t *testing.T) { "Could not unmarshal into baseFormat: %v", err) } - if !reflect.DeepEqual(newMsg, baseMsg) { + if !reflect.DeepEqual(*newMsg, baseMsg) { t.Errorf("unmarshalBaseFormat() error: "+ "Unmarshalled message does not match originally marshalled message."+ - "\n\tExpected: %v\n\tRecieved: %v", baseMsg, newMsg) + "\n\tExpected: %v\n\tRecieved: %v", baseMsg, *newMsg) } // Unmarshal error test: Invalid size parameter diff --git a/auth/manager.go b/auth/manager.go index 6be1c8bd1695af1ff5c5609ee6c58e4effea2b62..a2ba826141ccbcf3a25bbae0c1df00afbda42d79 100644 --- a/auth/manager.go +++ b/auth/manager.go @@ -102,7 +102,7 @@ func (m *Manager) RemoveSpecificConfirmCallback(id *id.ID) { } // Adds a general callback to be used on auth session renegotiations. -func (m *Manager) AddResetCallback(cb interfaces.ResetCallback) { +func (m *Manager) AddResetNotificationCallback(cb interfaces.ResetNotificationCallback) { m.resetCallbacks.AddOverride(cb) } diff --git a/backup/backup.go b/backup/backup.go index 866aa394ea525f7fd2af89a486365d7fa8579c10..b8b72f2563844980c21d8d0ebb9309b85c996221 100644 --- a/backup/backup.go +++ b/backup/backup.go @@ -53,6 +53,8 @@ type Backup struct { store *storage.Session backupContainer *interfaces.BackupContainer rng *fastRNG.StreamGenerator + + jsonParams string } // UpdateBackupFn is the callback that encrypted backup data is returned on @@ -141,6 +143,7 @@ func resumeBackup(updateBackupCb UpdateBackupFn, c *api.Client, store: store, backupContainer: backupContainer, rng: rng, + jsonParams: loadJson(store.GetKV()), } // Setting backup trigger in client @@ -208,6 +211,19 @@ func (b *Backup) TriggerBackup(reason string) { } } +func (b *Backup) AddJson(newJson string) { + b.mux.Lock() + defer b.mux.Unlock() + + if newJson != b.jsonParams { + b.jsonParams = newJson + if err := storeJson(newJson, b.store.GetKV()); err != nil { + jww.FATAL.Panicf("Failed to store json: %+v", err) + } + go b.TriggerBackup("New Json") + } +} + // StopBackup stops the backup processes and deletes the user's password, key, // salt, and parameters from storage. func (b *Backup) StopBackup() error { @@ -287,5 +303,8 @@ func (b *Backup) assembleBackup() backup.Backup { // Get contacts bu.Contacts.Identities = b.store.E2e().GetPartners() + //add the memoized json params + bu.JSONParams = b.jsonParams + return bu } diff --git a/backup/backup_test.go b/backup/backup_test.go index 2cf59bd022cb81bfbe3b3d2c58e1fdc84278fbc1..1dc785aa311e0916f06f71618975dc9afadb1c25 100644 --- a/backup/backup_test.go +++ b/backup/backup_test.go @@ -24,7 +24,7 @@ import ( // Tests that Backup.initializeBackup returns a new Backup with a copy of the // key and the callback. func Test_initializeBackup(t *testing.T) { - cbChan := make(chan []byte) + cbChan := make(chan []byte, 2) cb := func(encryptedBackup []byte) { cbChan <- encryptedBackup } expectedPassword := "MySuperSecurePassword" b, err := initializeBackup(expectedPassword, cb, nil, @@ -34,6 +34,12 @@ func Test_initializeBackup(t *testing.T) { t.Errorf("initializeBackup returned an error: %+v", err) } + select { + case <-cbChan: + case <-time.After(10 * time.Millisecond): + t.Error("Timed out waiting for callback.") + } + // Check that the correct password is in storage loadedPassword, err := loadPassword(b.store.GetKV()) if err != nil { @@ -89,6 +95,12 @@ func Test_resumeBackup(t *testing.T) { t.Errorf("Failed to initialize new Backup: %+v", err) } + select { + case <-cbChan1: + case <-time.After(10 * time.Millisecond): + t.Error("Timed out waiting for callback.") + } + // Get key and salt to compare to later key1, salt1, _, err := loadBackup(b.store.GetKV()) if err != nil { @@ -282,6 +294,82 @@ func TestBackup_IsBackupRunning(t *testing.T) { } } +func TestBackup_AddJson(t *testing.T) { + b := newTestBackup("MySuperSecurePassword", nil, t) + s := b.store + json := "{'data': {'one': 1}}" + + expectedCollatedBackup := backup.Backup{ + RegistrationTimestamp: s.GetUser().RegistrationTimestamp, + TransmissionIdentity: backup.TransmissionIdentity{ + RSASigningPrivateKey: s.GetUser().TransmissionRSA, + RegistrarSignature: s.User().GetTransmissionRegistrationValidationSignature(), + Salt: s.GetUser().TransmissionSalt, + ComputedID: s.GetUser().TransmissionID, + }, + ReceptionIdentity: backup.ReceptionIdentity{ + RSASigningPrivateKey: s.GetUser().ReceptionRSA, + RegistrarSignature: s.User().GetReceptionRegistrationValidationSignature(), + Salt: s.GetUser().ReceptionSalt, + ComputedID: s.GetUser().ReceptionID, + DHPrivateKey: s.GetUser().E2eDhPrivateKey, + DHPublicKey: s.GetUser().E2eDhPublicKey, + }, + UserDiscoveryRegistration: backup.UserDiscoveryRegistration{ + FactList: s.GetUd().GetFacts(), + }, + Contacts: backup.Contacts{Identities: s.E2e().GetPartners()}, + JSONParams: json, + } + + b.AddJson(json) + + collatedBackup := b.assembleBackup() + if !reflect.DeepEqual(expectedCollatedBackup, collatedBackup) { + t.Errorf("Collated backup does not match expected."+ + "\nexpected: %+v\nreceived: %+v", + expectedCollatedBackup, collatedBackup) + } +} + +func TestBackup_AddJson_badJson(t *testing.T) { + b := newTestBackup("MySuperSecurePassword", nil, t) + s := b.store + json := "abc{'i'm a bad json: 'one': 1'''}}" + + expectedCollatedBackup := backup.Backup{ + RegistrationTimestamp: s.GetUser().RegistrationTimestamp, + TransmissionIdentity: backup.TransmissionIdentity{ + RSASigningPrivateKey: s.GetUser().TransmissionRSA, + RegistrarSignature: s.User().GetTransmissionRegistrationValidationSignature(), + Salt: s.GetUser().TransmissionSalt, + ComputedID: s.GetUser().TransmissionID, + }, + ReceptionIdentity: backup.ReceptionIdentity{ + RSASigningPrivateKey: s.GetUser().ReceptionRSA, + RegistrarSignature: s.User().GetReceptionRegistrationValidationSignature(), + Salt: s.GetUser().ReceptionSalt, + ComputedID: s.GetUser().ReceptionID, + DHPrivateKey: s.GetUser().E2eDhPrivateKey, + DHPublicKey: s.GetUser().E2eDhPublicKey, + }, + UserDiscoveryRegistration: backup.UserDiscoveryRegistration{ + FactList: s.GetUd().GetFacts(), + }, + Contacts: backup.Contacts{Identities: s.E2e().GetPartners()}, + JSONParams: json, + } + + b.AddJson(json) + + collatedBackup := b.assembleBackup() + if !reflect.DeepEqual(expectedCollatedBackup, collatedBackup) { + t.Errorf("Collated backup does not match expected."+ + "\nexpected: %+v\nreceived: %+v", + expectedCollatedBackup, collatedBackup) + } +} + // Tests that Backup.assembleBackup returns the backup.Backup with the expected // results. func TestBackup_assembleBackup(t *testing.T) { diff --git a/backup/jsonStorage.go b/backup/jsonStorage.go new file mode 100644 index 0000000000000000000000000000000000000000..8ce778b56aff9c76847caf1953451d7cb6b39d5d --- /dev/null +++ b/backup/jsonStorage.go @@ -0,0 +1,30 @@ +package backup + +import ( + "gitlab.com/elixxir/client/storage/versioned" + "gitlab.com/xx_network/primitives/netTime" +) + +const ( + jsonStorageVersion = 0 + jsonStorageKey = "JsonStorage" +) + +func storeJson(json string, kv *versioned.KV) error { + obj := &versioned.Object{ + Version: jsonStorageVersion, + Timestamp: netTime.Now(), + Data: []byte(json), + } + + return kv.Set(jsonStorageKey, jsonStorageVersion, obj) +} + +func loadJson(kv *versioned.KV) string { + obj, err := kv.Get(jsonStorageKey, jsonStorageVersion) + if err != nil { + return "" + } + + return string(obj.Data) +} diff --git a/backup/jsonStorage_test.go b/backup/jsonStorage_test.go new file mode 100644 index 0000000000000000000000000000000000000000..d0207d910d76ffd73b5dfa1ff623996bf14c916b --- /dev/null +++ b/backup/jsonStorage_test.go @@ -0,0 +1,22 @@ +package backup + +import ( + "gitlab.com/elixxir/client/storage/versioned" + "gitlab.com/elixxir/ekv" + "testing" +) + +func Test_storeJson_loadJson(t *testing.T) { + kv := versioned.NewKV(make(ekv.Memstore)) + json := "{'data': {'one': 1}}" + + err := storeJson(json, kv) + if err != nil { + t.Errorf("Failed to store JSON: %+v", err) + } + + loaded := loadJson(kv) + if loaded != json { + t.Errorf("Did not receive expected data from KV.\n\tExpected: %s, Received: %s\n", json, loaded) + } +} diff --git a/bindings/authenticatedChannels.go b/bindings/authenticatedChannels.go index f52b9e7d6ddf4b0728b4499777cbd6e017632a09..af246b5408736224ec081ae7e45806b25b4a0a60 100644 --- a/bindings/authenticatedChannels.go +++ b/bindings/authenticatedChannels.go @@ -84,7 +84,7 @@ func (c *Client) ResetSession(recipientMarshaled, // RegisterAuthCallbacks registers all callbacks for authenticated channels. // This can only be called once func (c *Client) RegisterAuthCallbacks(request AuthRequestCallback, - confirm AuthConfirmCallback, reset AuthResetCallback) { + confirm AuthConfirmCallback, reset AuthResetNotificationCallback) { requestFunc := func(requestor contact.Contact) { requestorBind := &Contact{c: &requestor} @@ -103,7 +103,7 @@ func (c *Client) RegisterAuthCallbacks(request AuthRequestCallback, c.api.GetAuthRegistrar().AddGeneralConfirmCallback(confirmFunc) c.api.GetAuthRegistrar().AddGeneralRequestCallback(requestFunc) - c.api.GetAuthRegistrar().AddResetCallback(resetFunc) + c.api.GetAuthRegistrar().AddResetNotificationCallback(resetFunc) } // ConfirmAuthenticatedChannel creates an authenticated channel out of a valid diff --git a/bindings/backup.go b/bindings/backup.go index 9621fdfe69a2117a3939550490ae25fc3d5562dd..955839e8cf326ecda83e36fd35f2ba68fdb657f9 100644 --- a/bindings/backup.go +++ b/bindings/backup.go @@ -65,3 +65,8 @@ func (b *Backup) StopBackup() error { func (b *Backup) IsBackupRunning() bool { return b.b.IsBackupRunning() } + +// AddJson stores a passed in json string in the backup structure +func (b *Backup) AddJson(json string) { + b.b.AddJson(json) +} diff --git a/bindings/callback.go b/bindings/callback.go index 53ed8f1e7482f09827552a56975c4f4015c5fec9..4ad87c94ae91e1dc5d688900d75d0b404f9a1765 100644 --- a/bindings/callback.go +++ b/bindings/callback.go @@ -62,7 +62,7 @@ type AuthConfirmCallback interface { // AuthRequestCallback notifies the register whenever they receive an auth // request -type AuthResetCallback interface { +type AuthResetNotificationCallback interface { Callback(requestor *Contact) } diff --git a/bindings/client.go b/bindings/client.go index 1d27eebecf9dda16a8ed8b3e69f55ba4d52b4007..edda4a31dfa78239b4f7593f0a3db71af0a2021d 100644 --- a/bindings/client.go +++ b/bindings/client.go @@ -83,20 +83,30 @@ func NewPrecannedClient(precannedID int, network, storageDir string, password [] return nil } +type BackupReport struct { + RestoredContacts []*id.ID + Params string +} + // NewClientFromBackup constructs a new Client from an encrypted backup. The backup // is decrypted using the backupPassphrase. On success a successful client creation, // the function will return a JSON encoded list of the E2E partners -// contained in the backup. +// contained in the backup and a json-encoded string of the parameters stored in the backup func NewClientFromBackup(ndfJSON, storageDir string, sessionPassword, backupPassphrase, backupFileContents []byte) ([]byte, error) { - backupPartnerIds, err := api.NewClientFromBackup(ndfJSON, storageDir, + backupPartnerIds, jsonParams, err := api.NewClientFromBackup(ndfJSON, storageDir, sessionPassword, backupPassphrase, backupFileContents) if err != nil { return nil, errors.New(fmt.Sprintf("Failed to create new "+ "client from backup: %+v", err)) } - return json.Marshal(backupPartnerIds) + report := BackupReport{ + RestoredContacts: backupPartnerIds, + Params: jsonParams, + } + + return json.Marshal(report) } // Login will load an existing client from the storageDir diff --git a/bindings/restoreContacts.go b/bindings/restoreContacts.go index 61e14d3c1073e3e80a44fcbb58df4b6f400393ea..b970c77a2ed8cbf5664f76a6754904edb9b8d0f0 100644 --- a/bindings/restoreContacts.go +++ b/bindings/restoreContacts.go @@ -29,6 +29,7 @@ type RestoreContactsReport struct { restored []*id.ID failed []*id.ID errs []error + restErr error } // LenRestored returns the length of ID's restored. @@ -56,6 +57,14 @@ func (r *RestoreContactsReport) GetErrorAt(index int) string { return r.errs[index].Error() } +// GetRestoreContactsError returns an error string. Empty if no error. +func (r *RestoreContactsReport) GetRestoreContactsError() string { + if r.restErr == nil { + return "" + } + return r.restErr.Error() +} + // RestoreContactsFromBackup takes as input the jason output of the // `NewClientFromBackup` function, unmarshals it into IDs, looks up // each ID in user discovery, and initiates a session reset request. @@ -65,8 +74,8 @@ func (r *RestoreContactsReport) GetErrorAt(index int) string { // the mobile phone apps and are not intended to be part of the xxDK. It // should be treated as internal functions specific to the phone apps. func RestoreContactsFromBackup(backupPartnerIDs []byte, client *Client, - udManager *UserDiscovery, updatesCb RestoreContactsUpdater) ( - *RestoreContactsReport, error) { + udManager *UserDiscovery, + updatesCb RestoreContactsUpdater) *RestoreContactsReport { restored, failed, errs, err := xxmutils.RestoreContactsFromBackup( backupPartnerIDs, &client.api, udManager.ud, updatesCb) @@ -75,6 +84,7 @@ func RestoreContactsFromBackup(backupPartnerIDs []byte, client *Client, restored: restored, failed: failed, errs: errs, - }, err + restErr: err, + } } diff --git a/bindings/url.go b/bindings/url.go index 0322b6c3dd05318bbce6a1d9dee8ed9caeebf7da..f561540f3ecf89fb1b67e139549208884fe46779 100644 --- a/bindings/url.go +++ b/bindings/url.go @@ -7,10 +7,9 @@ package bindings - import ( - "gitlab.com/xx_network/primitives/id" "fmt" + "gitlab.com/xx_network/primitives/id" ) const dashboardBaseURL = "https://dashboard.xx.network" diff --git a/cmd/root.go b/cmd/root.go index 28e3a988faa5a7613644d6f18c82a76205ac80b7..d7bb3d6f5c65399751fb00a454752cc103c70902 100644 --- a/cmd/root.go +++ b/cmd/root.go @@ -525,7 +525,7 @@ func createClient() *api.Client { initLog(logLevel, viper.GetString("log")) jww.INFO.Printf(Version()) - pass := viper.GetString("password") + pass := parsePassword(viper.GetString("password")) storeDir := viper.GetString("session") regCode := viper.GetString("regcode") precannedID := viper.GetUint("sendid") @@ -544,17 +544,17 @@ func createClient() *api.Client { if precannedID != 0 { err = api.NewPrecannedClient(precannedID, - string(ndfJSON), storeDir, []byte(pass)) + string(ndfJSON), storeDir, pass) } else if protoUserPath != "" { protoUserJson, err := utils.ReadFile(protoUserPath) if err != nil { jww.FATAL.Panicf("%v", err) } err = api.NewProtoClient_Unsafe(string(ndfJSON), storeDir, - []byte(pass), protoUserJson) + pass, protoUserJson) } else if userIDprefix != "" { err = api.NewVanityClient(string(ndfJSON), storeDir, - []byte(pass), regCode, userIDprefix) + pass, regCode, userIDprefix) } else if backupPath != "" { b, backupFile := loadBackup(backupPath, string(backupPass)) @@ -572,8 +572,8 @@ func createClient() *api.Client { } // Construct client from backup data - backupIdList, err := api.NewClientFromBackup(string(ndfJSON), storeDir, - []byte(pass), backupPass, backupFile) + backupIdList, _, err := api.NewClientFromBackup(string(ndfJSON), storeDir, + pass, backupPass, backupFile) backupIdListPath := viper.GetString("backupIdList") if backupIdListPath != "" { @@ -593,7 +593,7 @@ func createClient() *api.Client { } else { err = api.NewClient(string(ndfJSON), storeDir, - []byte(pass), regCode) + pass, regCode) } if err != nil { @@ -612,7 +612,7 @@ func createClient() *api.Client { netParams.ForceMessagePickupRetry = viper.GetBool("forceMessagePickupRetry") netParams.VerboseRoundTracking = viper.GetBool("verboseRoundTracking") - client, err := api.OpenClient(storeDir, []byte(pass), netParams) + client, err := api.OpenClient(storeDir, pass, netParams) if err != nil { jww.FATAL.Panicf("%+v", err) } @@ -622,7 +622,7 @@ func createClient() *api.Client { func initClient() *api.Client { createClient() - pass := viper.GetString("password") + pass := parsePassword(viper.GetString("password")) storeDir := viper.GetString("session") jww.DEBUG.Printf("sessionDur: %v", storeDir) netParams := params.GetDefaultNetwork() @@ -642,7 +642,7 @@ func initClient() *api.Client { netParams.VerboseRoundTracking = viper.GetBool("verboseRoundTracking") // load the client - client, err := api.Login(storeDir, []byte(pass), netParams) + client, err := api.Login(storeDir, pass, netParams) if err != nil { jww.FATAL.Panicf("%+v", err) } @@ -853,6 +853,16 @@ func getPrecanID(recipientID *id.ID) uint { return uint(recipientID.Bytes()[7]) } +func parsePassword(pwStr string) []byte { + if strings.HasPrefix(pwStr, "0x") { + return getPWFromHexString(pwStr[2:]) + } else if strings.HasPrefix(pwStr, "b64:") { + return getPWFromb64String(pwStr[4:]) + } else { + return []byte(pwStr) + } +} + func parseRecipient(idStr string) (*id.ID, bool) { if idStr == "0" { return nil, false @@ -905,6 +915,23 @@ func getUIDFromb64String(idStr string) *id.ID { return ID } +func getPWFromHexString(pwStr string) []byte { + pwBytes, err := hex.DecodeString(fmt.Sprintf("%0*d%s", + 66-len(pwStr), 0, pwStr)) + if err != nil { + jww.FATAL.Panicf("%+v", err) + } + return pwBytes +} + +func getPWFromb64String(pwStr string) []byte { + pwBytes, err := base64.StdEncoding.DecodeString(pwStr) + if err != nil { + jww.FATAL.Panicf("%+v", err) + } + return pwBytes +} + func getUIDFromString(idStr string) *id.ID { idInt, err := strconv.Atoi(idStr) if err != nil { diff --git a/cmd/version.go b/cmd/version.go index f6443702ed89544cfeda43c6671e75d740bb21fc..78dee4c0b41830b5f6df795bb4e55192a3c037b5 100644 --- a/cmd/version.go +++ b/cmd/version.go @@ -18,7 +18,7 @@ import ( ) // Change this value to set the version for this build -const currentVersion = "4.0.0" +const currentVersion = "4.1.0" func Version() string { out := fmt.Sprintf("Elixxir Client v%s -- %s\n\n", api.SEMVER, diff --git a/go.mod b/go.mod index d21814b72e803fcee02798ea1f6c1b288361e2a8..0d5e6d485cfc66aa506a1fefa30f30cbf1c3999c 100644 --- a/go.mod +++ b/go.mod @@ -12,11 +12,11 @@ require ( github.com/spf13/jwalterweatherman v1.1.0 github.com/spf13/viper v1.7.1 gitlab.com/elixxir/bloomfilter v0.0.0-20200930191214-10e9ac31b228 - gitlab.com/elixxir/comms v0.0.4-0.20220302214218-7cef54bc0c37 - gitlab.com/elixxir/crypto v0.0.7-0.20220222221347-95c7ae58da6b + gitlab.com/elixxir/comms v0.0.4-0.20220308183624-c2183e687a03 + gitlab.com/elixxir/crypto v0.0.7-0.20220309234716-1ba339865787 gitlab.com/elixxir/ekv v0.1.6 gitlab.com/elixxir/primitives v0.0.3-0.20220222212109-d412a6e46623 - gitlab.com/xx_network/comms v0.0.4-0.20220223205228-7c4974139569 + gitlab.com/xx_network/comms v0.0.4-0.20220311192415-d95fe8906580 gitlab.com/xx_network/crypto v0.0.5-0.20220222212031-750f7e8a01f4 gitlab.com/xx_network/primitives v0.0.4-0.20220222211843-901fa4a2d72b golang.org/x/crypto v0.0.0-20220128200615-198e4374d7ed diff --git a/go.sum b/go.sum index 9ab0c958936ee3dda207fc2f11880a93d4b26441..d5504c7a5d85bda9142d40c5dd364953fd617a98 100644 --- a/go.sum +++ b/go.sum @@ -272,12 +272,13 @@ github.com/zeebo/pcg v1.0.0 h1:dt+dx+HvX8g7Un32rY9XWoYnd0NmKmrIzpHF7qiTDj0= github.com/zeebo/pcg v1.0.0/go.mod h1:09F0S9iiKrwn9rlI5yjLkmrug154/YRW6KnnXVDM/l4= gitlab.com/elixxir/bloomfilter v0.0.0-20200930191214-10e9ac31b228 h1:Gi6rj4mAlK0BJIk1HIzBVMjWNjIUfstrsXC2VqLYPcA= gitlab.com/elixxir/bloomfilter v0.0.0-20200930191214-10e9ac31b228/go.mod h1:H6jztdm0k+wEV2QGK/KYA+MY9nj9Zzatux/qIvDDv3k= -gitlab.com/elixxir/comms v0.0.4-0.20220302214218-7cef54bc0c37 h1:VifEG3xpMq8bKkOyIL3KvEeyWCBmPvadKc7Mss5TZ1A= -gitlab.com/elixxir/comms v0.0.4-0.20220302214218-7cef54bc0c37/go.mod h1:4yMdU+Jee5W9lqkZGHJAuipEhW7FloT0eyVEFUJza+E= +gitlab.com/elixxir/comms v0.0.4-0.20220308183624-c2183e687a03 h1:4eNjO3wCyHgxpGeq2zgDb5SsdTcQaG5IZjBOuEL6KgM= +gitlab.com/elixxir/comms v0.0.4-0.20220308183624-c2183e687a03/go.mod h1:4yMdU+Jee5W9lqkZGHJAuipEhW7FloT0eyVEFUJza+E= gitlab.com/elixxir/crypto v0.0.0-20200804182833-984246dea2c4/go.mod h1:ucm9SFKJo+K0N2GwRRpaNr+tKXMIOVWzmyUD0SbOu2c= gitlab.com/elixxir/crypto v0.0.3/go.mod h1:ZNgBOblhYToR4m8tj4cMvJ9UsJAUKq+p0gCp07WQmhA= -gitlab.com/elixxir/crypto v0.0.7-0.20220222221347-95c7ae58da6b h1:m80Ub5mshPbMzYjRC0nXuI8vtm6e5crISczRsP2YUJ4= gitlab.com/elixxir/crypto v0.0.7-0.20220222221347-95c7ae58da6b/go.mod h1:tD6XjtQh87T2nKZL5I/pYPck5M2wLpkZ1Oz7H/LqO10= +gitlab.com/elixxir/crypto v0.0.7-0.20220309234716-1ba339865787 h1:+qmsWov412+Yn7AKUhTbOcDgAydNXlNLPmFpO2W5LwY= +gitlab.com/elixxir/crypto v0.0.7-0.20220309234716-1ba339865787/go.mod h1:tD6XjtQh87T2nKZL5I/pYPck5M2wLpkZ1Oz7H/LqO10= gitlab.com/elixxir/ekv v0.1.6 h1:M2hUSNhH/ChxDd+s8xBqSEKgoPtmE6hOEBqQ73KbN6A= gitlab.com/elixxir/ekv v0.1.6/go.mod h1:e6WPUt97taFZe5PFLPb1Dupk7tqmDCTQu1kkstqJvw4= gitlab.com/elixxir/primitives v0.0.0-20200731184040-494269b53b4d/go.mod h1:OQgUZq7SjnE0b+8+iIAT2eqQF+2IFHn73tOo+aV11mg= @@ -287,8 +288,9 @@ gitlab.com/elixxir/primitives v0.0.1/go.mod h1:kNp47yPqja2lHSiS4DddTvFpB/4D9dB2Y gitlab.com/elixxir/primitives v0.0.3-0.20220222212109-d412a6e46623 h1:NzJ06KdJd3fVJee0QvGhNr3CO+Ki8Ea1PeakZsm+rZM= gitlab.com/elixxir/primitives v0.0.3-0.20220222212109-d412a6e46623/go.mod h1:MtFIyJUQn9P7djzVlBpEYkPNnnWFTjZvw89swoXY+QM= gitlab.com/xx_network/comms v0.0.0-20200805174823-841427dd5023/go.mod h1:owEcxTRl7gsoM8c3RQ5KAm5GstxrJp5tn+6JfQ4z5Hw= -gitlab.com/xx_network/comms v0.0.4-0.20220223205228-7c4974139569 h1:TjO165sJ6S++ZcHn+u5GnkfCjq3sxfaqhHmB1WgH31s= gitlab.com/xx_network/comms v0.0.4-0.20220223205228-7c4974139569/go.mod h1:isHnwem0v4rTcwwHP455FhVlFyPcHkHiVz+N3s/uCSI= +gitlab.com/xx_network/comms v0.0.4-0.20220311192415-d95fe8906580 h1:IV0gDwdTxtCpc9Vkx7IeSStSqvG+0ZpF57X+OhTQDIM= +gitlab.com/xx_network/comms v0.0.4-0.20220311192415-d95fe8906580/go.mod h1:isHnwem0v4rTcwwHP455FhVlFyPcHkHiVz+N3s/uCSI= gitlab.com/xx_network/crypto v0.0.3/go.mod h1:DF2HYvvCw9wkBybXcXAgQMzX+MiGbFPjwt3t17VRqRE= gitlab.com/xx_network/crypto v0.0.4/go.mod h1:+lcQEy+Th4eswFgQDwT0EXKp4AXrlubxalwQFH5O0Mk= gitlab.com/xx_network/crypto v0.0.5-0.20220222212031-750f7e8a01f4 h1:95dZDMn/hpLNwsgZO9eyQgGKaSDyh6F6+WygqZIciww= diff --git a/groupChat/send.go b/groupChat/send.go index 03c513ccc032ab28778dfb94bb44d05c1362355e..f5bf054fb2cb3b318ab61d15912001f1507af23f 100644 --- a/groupChat/send.go +++ b/groupChat/send.go @@ -73,7 +73,7 @@ func (m *Manager) createMessages(groupID *id.ID, msg []byte, timestamp time.Time cmixMsg := format.NewMessage(m.store.Cmix().GetGroup().GetP().ByteLen()) _, intlMsg, err := newMessageParts(cmixMsg.ContentsSize()) if err != nil { - return nil, group.MessageID{},errors.WithMessage(err,"Failed to make message parts for message ID") + return nil, group.MessageID{}, errors.WithMessage(err, "Failed to make message parts for message ID") } messageID := group.NewMessageID(groupID, setInternalPayload(intlMsg, timestamp, m.gs.GetUser().ID, msg)) diff --git a/groupChat/sendRequests.go b/groupChat/sendRequests.go index 555fd2c5e0cfc891ba8cf37877e6dc72353aebf3..6318e6420cf558034c9ac1dfa36f0db715ba677d 100644 --- a/groupChat/sendRequests.go +++ b/groupChat/sendRequests.go @@ -117,10 +117,9 @@ func (m Manager) sendRequest(memberID *id.ID, request []byte) ([]id.Round, error MessageType: message.GroupCreationRequest, } - recipent, err := m.store.E2e().GetPartner(memberID) - if err!=nil{ - return nil, errors.WithMessagef(err,"Failed to send request to %s " + + if err != nil { + return nil, errors.WithMessagef(err, "Failed to send request to %s "+ "because e2e relationship could not be found", memberID) } diff --git a/groupChat/sendRequests_test.go b/groupChat/sendRequests_test.go index 9f2b9c19f2eb2c65dd51d3f66e3dc512cfe991b3..c5ec225a4d4638e518fca793da97bdfabfea5162 100644 --- a/groupChat/sendRequests_test.go +++ b/groupChat/sendRequests_test.go @@ -38,7 +38,7 @@ func TestManager_ResendRequest(t *testing.T) { Created: g.Created.UnixNano(), } - for i := range g.Members{ + for i := range g.Members { grp := m.store.E2e().GetGroup() dhKey := grp.NewInt(int64(i + 42)) pubKey := diffieHellman.GeneratePublicKey(dhKey, grp) @@ -60,7 +60,6 @@ func TestManager_ResendRequest(t *testing.T) { t.Errorf("ResendRequest() returned an error: %+v", err) } - if status != AllSent { t.Errorf("ResendRequest() failed to return the expected status."+ "\nexpected: %s\nreceived: %s", AllSent, status) @@ -135,7 +134,7 @@ func TestManager_sendRequests(t *testing.T) { Created: g.Created.UnixNano(), } - for i := range g.Members{ + for i := range g.Members { grp := m.store.E2e().GetGroup() dhKey := grp.NewInt(int64(i + 42)) pubKey := diffieHellman.GeneratePublicKey(dhKey, grp) @@ -235,7 +234,7 @@ func TestManager_sendRequests_SendPartialSent(t *testing.T) { expectedErr := fmt.Sprintf(sendRequestPartialErr, (len(g.Members)-1)/2, len(g.Members)-1, "") - for i := range g.Members{ + for i := range g.Members { grp := m.store.E2e().GetGroup() dhKey := grp.NewInt(int64(i + 42)) pubKey := diffieHellman.GeneratePublicKey(dhKey, grp) @@ -274,7 +273,7 @@ func TestManager_sendRequest(t *testing.T) { prng := rand.New(rand.NewSource(42)) m, g := newTestManagerWithStore(prng, 10, 0, nil, nil, t) - for i := range g.Members{ + for i := range g.Members { grp := m.store.E2e().GetGroup() dhKey := grp.NewInt(int64(i + 42)) pubKey := diffieHellman.GeneratePublicKey(dhKey, grp) @@ -332,7 +331,6 @@ func TestManager_sendRequest_SendE2eError(t *testing.T) { t.Errorf("Failed to add partner %s: %+v", recipientID, err) } - _, err = m.sendRequest(recipientID, nil) if err == nil || !strings.Contains(err.Error(), expectedErr) { t.Errorf("sendRequest() failed to return the expected error."+ diff --git a/interfaces/auth.go b/interfaces/auth.go index 28443cf800727ad20cd9ff9586dcb5945e0a87e5..d82625c723150b433959d22c83241c909fb3781e 100644 --- a/interfaces/auth.go +++ b/interfaces/auth.go @@ -14,7 +14,7 @@ import ( type RequestCallback func(requestor contact.Contact) type ConfirmCallback func(partner contact.Contact) -type ResetCallback func(partner contact.Contact) +type ResetNotificationCallback func(partner contact.Contact) type Auth interface { // Adds a general callback to be used on auth requests. This will be preempted @@ -43,8 +43,8 @@ type Auth interface { AddSpecificConfirmCallback(id *id.ID, cb ConfirmCallback) // Removes a specific callback to be used on auth confirm. RemoveSpecificConfirmCallback(id *id.ID) - // Add a callback to receive session renegotiations - AddResetCallback(cb ResetCallback) + // Add a callback to receive session renegotiation notifications + AddResetNotificationCallback(cb ResetNotificationCallback) //Replays all pending received requests over tha callbacks ReplayRequests() } diff --git a/interfaces/params/E2E.go b/interfaces/params/E2E.go index 66f5b8f989468ba4988dfeefd7dc00cd5d438510..f3aed87855d884b9d99f8de68af9bac5a6a2ca1d 100644 --- a/interfaces/params/E2E.go +++ b/interfaces/params/E2E.go @@ -63,38 +63,36 @@ func (st SendType) String() string { // Network E2E Params - - type E2ESessionParams struct { // using the DH as a seed, both sides generate a number // of keys to use before they must rekey because // there are no keys to use. - MinKeys uint16 - MaxKeys uint16 + MinKeys uint16 + MaxKeys uint16 // the percent of keys before a rekey is attempted. must be <0 - RekeyThreshold float64 + RekeyThreshold float64 // extra keys generated and reserved for rekey attempts. This // many keys are not allowed to be used for sending messages // in order to ensure there are extras for rekeying. - NumRekeys uint16 + NumRekeys uint16 } // DEFAULT KEY GENERATION PARAMETERS // Hardcoded limits for keys // sets the number of keys very high, but with a low rekey threshold. In this case, if the other party is online, you will read const ( - minKeys uint16 = 1000 - maxKeys uint16 = 2000 - rekeyThrshold float64 = 0.05 - numReKeys uint16 = 16 + minKeys uint16 = 1000 + maxKeys uint16 = 2000 + rekeyThrshold float64 = 0.05 + numReKeys uint16 = 16 ) func GetDefaultE2ESessionParams() E2ESessionParams { return E2ESessionParams{ - MinKeys: minKeys, - MaxKeys: maxKeys, + MinKeys: minKeys, + MaxKeys: maxKeys, RekeyThreshold: rekeyThrshold, - NumRekeys: numReKeys, + NumRekeys: numReKeys, } } diff --git a/interfaces/params/message.go b/interfaces/params/message.go index 27a8ebd7d626445cc4c2ace03772d5dbbb6105ed..66371a7797d1c647c3a7efe96d801f4980e008fd 100644 --- a/interfaces/params/message.go +++ b/interfaces/params/message.go @@ -16,7 +16,7 @@ type Messages struct { MessageReceptionWorkerPoolSize uint MaxChecksGarbledMessage uint GarbledMessageWait time.Duration - RealtimeOnly bool + RealtimeOnly bool } func GetDefaultMessage() Messages { @@ -25,6 +25,6 @@ func GetDefaultMessage() Messages { MessageReceptionWorkerPoolSize: 4, MaxChecksGarbledMessage: 10, GarbledMessageWait: 15 * time.Minute, - RealtimeOnly: false, + RealtimeOnly: false, } } diff --git a/interfaces/params/network.go b/interfaces/params/network.go index 7e3b6f4ab3f72f664af8e5b7e9cc5d226950a37f..a89db50e17a1063d99ef17e26cd12ab6fcc9b24f 100644 --- a/interfaces/params/network.go +++ b/interfaces/params/network.go @@ -71,7 +71,7 @@ func (n Network) Marshal() ([]byte, error) { return json.Marshal(n) } -func (n Network) SetRealtimeOnlyAll()Network { +func (n Network) SetRealtimeOnlyAll() Network { n.RealtimeOnly = true n.Rounds.RealtimeOnly = true n.Messages.RealtimeOnly = true diff --git a/keyExchange/rekey.go b/keyExchange/rekey.go index 04bb31413af965f621884a01ab0f557c632ec504..ea14e92fa37bda8a2826eca10e43112fc1661530 100644 --- a/keyExchange/rekey.go +++ b/keyExchange/rekey.go @@ -164,8 +164,8 @@ func negotiate(instance *network.Instance, sendE2E interfaces.SendE2E, session, msgID) err = session.TrySetNegotiationStatus(e2e.Sent) if err != nil { - if (session.NegotiationStatus() == e2e.NewSessionTriggered) { - msg := fmt.Sprintf("All channels exhausted for %s, " + + if session.NegotiationStatus() == e2e.NewSessionTriggered { + msg := fmt.Sprintf("All channels exhausted for %s, "+ "rekey impossible.", session) return errors.WithMessage(err, msg) } diff --git a/network/follow.go b/network/follow.go index 6f98a33b50127d78f10e703b1c75081d02357d0f..817ba0effc0c01e744cd430effa7ecc874d746fb 100644 --- a/network/follow.go +++ b/network/follow.go @@ -372,7 +372,7 @@ func (m *manager) follow(report interfaces.ClientErrorReport, rng csprng.Source, var roundsWithMessages2 []id.Round - if !m.param.RealtimeOnly{ + if !m.param.RealtimeOnly { roundsWithMessages2 = identity.UR.Iterate(func(rid id.Round) bool { if gwRoundsState.Checked(rid) { return rounds.Checker(rid, filterList, identity.CR) @@ -381,7 +381,6 @@ func (m *manager) follow(report interfaces.ClientErrorReport, rng csprng.Source, }, roundsUnknown, abandon) } - for _, rid := range roundsWithMessages { //denote that the round has been looked at in the tracking store if identity.CR.Check(rid) { diff --git a/network/gateway/hostPool.go b/network/gateway/hostPool.go index eed695bb9a56ebc4ef8ef4e43289e19376593b3f..dd4616cc4583d54621cf2fb42dc2611ab96cee95 100644 --- a/network/gateway/hostPool.go +++ b/network/gateway/hostPool.go @@ -74,21 +74,23 @@ type HostPool struct { // PoolParams Allows configuration of HostPool parameters type PoolParams struct { - MaxPoolSize uint32 // Maximum number of Hosts in the HostPool - PoolSize uint32 // Allows override of HostPool size. Set to zero for dynamic size calculation - ProxyAttempts uint32 // How many proxies will be used in event of send failure - MaxPings uint32 // How many gateways to concurrently test when initializing HostPool. Disabled if zero. - HostParams connect.HostParams // Parameters for the creation of new Host objects + MaxPoolSize uint32 // Maximum number of Hosts in the HostPool + PoolSize uint32 // Allows override of HostPool size. Set to zero for dynamic size calculation + ProxyAttempts uint32 // How many proxies will be used in event of send failure + MaxPings uint32 // How many gateways to concurrently test when initializing HostPool. Disabled if zero. + ForceConnection bool // Flag determining whether Host connections are initialized when added to HostPool + HostParams connect.HostParams // Parameters for the creation of new Host objects } // DefaultPoolParams Returns a default set of PoolParams func DefaultPoolParams() PoolParams { p := PoolParams{ - MaxPoolSize: 30, - ProxyAttempts: 5, - PoolSize: 0, - MaxPings: 0, - HostParams: connect.GetDefaultHostParams(), + MaxPoolSize: 30, + ProxyAttempts: 5, + PoolSize: 0, + MaxPings: 0, + ForceConnection: false, + HostParams: connect.GetDefaultHostParams(), } p.HostParams.MaxRetries = 1 p.HostParams.MaxSendRetries = 1 @@ -539,7 +541,7 @@ func (h *HostPool) replaceHostNoStore(newId *id.ID, oldPoolIndex uint32) error { // Use the GwId to keep track of the new random Host's index in the hostList h.hostMap[*newId] = oldPoolIndex - // Clean up and move onto next Host + // Clean up and disconnect old Host oldHostIDStr := "unknown" if oldHost != nil { oldHostIDStr = oldHost.GetId().String() @@ -547,9 +549,18 @@ func (h *HostPool) replaceHostNoStore(newId *id.ID, oldPoolIndex uint32) error { go oldHost.Disconnect() } + // Manually connect the new Host + if h.poolParams.ForceConnection { + go func() { + err := newHost.Connect() + if err != nil { + jww.WARN.Printf("Unable to initialize Host connection: %+v", err) + } + }() + } + jww.DEBUG.Printf("Replaced Host at %d [%s] with new Host %s", oldPoolIndex, oldHostIDStr, newId.String()) - return nil } @@ -653,7 +664,6 @@ func (h *HostPool) addGateway(gwId *id.ID, ndfIndex int) { // Check if the host exists host, ok := h.manager.GetHost(gwId) if !ok { - // Check if gateway ID collides with an existing hard coded ID if id.CollidesWithHardCodedID(gwId) { jww.ERROR.Printf("Gateway ID invalid, collides with a "+ @@ -714,8 +724,8 @@ func readUint32(rng io.Reader) uint32 { func readRangeUint32(start, end uint32, rng io.Reader) uint32 { size := end - start // note we could just do the part inside the () here, but then extra - // can == size which means a little bit of range is wastes, either - // choice seems negligible so we went with the "more correct" + // can == size which means a little range is wasted, either + // choice seems negligible, so we went with the "more correct" extra := (math.MaxUint32%size + 1) % size limit := math.MaxUint32 - extra // Loop until we read something inside the limit diff --git a/network/gateway/hostpool_test.go b/network/gateway/hostpool_test.go index 5e93ba8df80ded94f70034bdc6ba3540ac6d1644..30f3b6a6151626a68e5d2b2fe4d3c282de0aefeb 100644 --- a/network/gateway/hostpool_test.go +++ b/network/gateway/hostpool_test.go @@ -485,11 +485,12 @@ func TestHostPool_UpdateNdf(t *testing.T) { // Construct a manager (bypass business logic in constructor) hostPool := &HostPool{ - manager: manager, - hostList: make([]*connect.Host, newIndex+1), - hostMap: make(map[id.ID]uint32), - ndf: testNdf, - storage: storage.InitTestingSession(t), + manager: manager, + hostList: make([]*connect.Host, newIndex+1), + hostMap: make(map[id.ID]uint32), + ndf: testNdf, + storage: storage.InitTestingSession(t), + poolParams: DefaultPoolParams(), filter: func(m map[id.ID]int, _ *ndf.NetworkDefinition) map[id.ID]int { return m }, @@ -855,6 +856,7 @@ func TestHostPool_AddGateway(t *testing.T) { hostList: make([]*connect.Host, newIndex+1), hostMap: make(map[id.ID]uint32), ndf: testNdf, + poolParams: params, addGatewayChan: make(chan network.NodeGateway), storage: storage.InitTestingSession(t), } @@ -888,6 +890,7 @@ func TestHostPool_RemoveGateway(t *testing.T) { hostList: make([]*connect.Host, newIndex+1), hostMap: make(map[id.ID]uint32), ndf: testNdf, + poolParams: params, addGatewayChan: make(chan network.NodeGateway), storage: storage.InitTestingSession(t), rng: fastRNG.NewStreamGenerator(1, 1, csprng.NewSystemRNG), diff --git a/network/manager.go b/network/manager.go index bba264c6b63263a196d4bfc135f1db42649948ec..1cec837978ffc0aaa120ea061b4d993df8110e07 100644 --- a/network/manager.go +++ b/network/manager.go @@ -133,6 +133,7 @@ func NewManager(session *storage.Session, switchboard *switchboard.Switchboard, poolParams.HostParams.KaClientOpts.Time = time.Duration(math.MaxInt64) // Enable optimized HostPool initialization poolParams.MaxPings = 50 + poolParams.ForceConnection = true m.sender, err = gateway.NewSender(poolParams, rng, ndf, comms, session, m.NodeRegistration) if err != nil { diff --git a/network/message/garbled_test.go b/network/message/garbled_test.go index 4a0f7eda9cb50fe7e7ed61aee72040d3756edb10..86f23b84776abbf2d2e9eae52fec21a690540ad9 100644 --- a/network/message/garbled_test.go +++ b/network/message/garbled_test.go @@ -152,7 +152,7 @@ func TestManager_CheckGarbledMessages(t *testing.T) { copy(fmp.Timestamp, ts) msg.SetContents(fmp.Bytes()) encryptedMsg := key.Encrypt(msg) - msg.SetIdentityFP(fingerprint.IdentityFP( msg.GetContents(), preimage.Data)) + msg.SetIdentityFP(fingerprint.IdentityFP(msg.GetContents(), preimage.Data)) i.Session.GetGarbledMessages().Add(encryptedMsg) stop := stoppable.NewSingle("stop") diff --git a/network/message/utils_test.go b/network/message/utils_test.go index 80b31c9a911a49babe5862078fc864498f3fdb32..e0bf2222d6f36a6934fd3469cfa1222fe257bae1 100644 --- a/network/message/utils_test.go +++ b/network/message/utils_test.go @@ -17,10 +17,10 @@ func (mc *MockSendCMIXComms) GetHost(*id.ID) (*connect.Host, bool) { nid1 := id.NewIdFromString("zezima", id.Node, mc.t) gwID := nid1.DeepCopy() gwID.SetType(id.Gateway) - h, _ := connect.NewHost(gwID, "0.0.0.0", []byte(""), connect.HostParams{ - MaxRetries: 0, - AuthEnabled: false, - }) + p := connect.GetDefaultHostParams() + p.MaxRetries = 0 + p.AuthEnabled = false + h, _ := connect.NewHost(gwID, "0.0.0.0", []byte(""), p) return h, true } diff --git a/network/node/register.go b/network/node/register.go index 274703a8d1fb68f001e0ab692a8c726243ed38dc..0a741894cb2c5f0f60d530006ed6db387eff050f 100644 --- a/network/node/register.go +++ b/network/node/register.go @@ -39,9 +39,9 @@ import ( "time" ) - const maxAttempts = 5 -var delayTable = [5]time.Duration{0,5*time.Second,30*time.Second,60*time.Second,120*time.Second} + +var delayTable = [5]time.Duration{0, 5 * time.Second, 30 * time.Second, 60 * time.Second, 120 * time.Second} type RegisterNodeCommsInterface interface { SendRequestClientKeyMessage(host *connect.Host, @@ -53,7 +53,6 @@ func StartRegistration(sender *gateway.Sender, session *storage.Session, rngGen multi := stoppable.NewMulti("NodeRegistrations") - inProgess := &sync.Map{} // we are relying on the in progress check to // ensure there is only a single operator at a time, as a result this is a map of ID -> int @@ -96,7 +95,7 @@ func registerNodes(sender *gateway.Sender, session *storage.Session, //keep track of how many times this has been attempted numAttempts := uint(1) - if nunAttemptsInterface, hasValue := attempts.LoadOrStore(nidStr, numAttempts); hasValue{ + if nunAttemptsInterface, hasValue := attempts.LoadOrStore(nidStr, numAttempts); hasValue { numAttempts = nunAttemptsInterface.(uint) attempts.Store(nidStr, numAttempts+1) } @@ -112,11 +111,11 @@ func registerNodes(sender *gateway.Sender, session *storage.Session, if err != nil { jww.ERROR.Printf("Failed to register node: %+v", err) //if we have not reached the attempt limit for this gateway, send it back into the channel to retry - if numAttempts <maxAttempts{ - go func(){ + if numAttempts < maxAttempts { + go func() { //delay the send for a backoff time.Sleep(delayTable[numAttempts-1]) - c<-gw + c <- gw }() } } @@ -125,8 +124,6 @@ func registerNodes(sender *gateway.Sender, session *storage.Session, } } - - //registerWithNode serves as a helper for RegisterWithNodes // It registers a user with a specific in the client's ndf. func registerWithNode(sender *gateway.Sender, comms RegisterNodeCommsInterface, @@ -181,7 +178,6 @@ func requestKey(sender *gateway.Sender, comms RegisterNodeCommsInterface, uci *user.CryptographicIdentity, store *cmix.Store, rng csprng.Source, stop *stoppable.Single) (*cyclic.Int, []byte, uint64, error) { - grp := store.GetGroup() // FIXME: Why 256 bits? -- this is spec but not explained, it has diff --git a/network/rounds/manager.go b/network/rounds/manager.go index f220583de8de98ab706b48db3a2b24d6a6168078..43d1f5f5a1e9acc4b09fc360d583d16e7e73c51e 100644 --- a/network/rounds/manager.go +++ b/network/rounds/manager.go @@ -58,12 +58,11 @@ func (m *Manager) StartProcessors() stoppable.Stoppable { } // Start the periodic unchecked round worker - if !m.params.RealtimeOnly{ + if !m.params.RealtimeOnly { stopper := stoppable.NewSingle("UncheckRound") go m.processUncheckedRounds(m.params.UncheckRoundPeriod, backOffTable, stopper) multi.Add(stopper) } - return multi } diff --git a/network/rounds/remoteFilters_test.go b/network/rounds/remoteFilters_test.go index 51d26973d9f024142ed89b04e57d8f7117992268..e490432857d5e94bf599d3b7eb3c4b1a66ee8017 100644 --- a/network/rounds/remoteFilters_test.go +++ b/network/rounds/remoteFilters_test.go @@ -20,7 +20,7 @@ import ( ) func TestMain(m *testing.M) { - jww.SetStdoutThreshold(jww.LevelTrace) + jww.SetStdoutThreshold(jww.LevelDebug) connect.TestingOnlyDisableTLS = true os.Exit(m.Run()) } @@ -98,4 +98,4 @@ func TestRemoteFilter_FirstLastRound(t *testing.T) { "\n\tExpected: %v\n\tReceived: %v", receivedLastRound, firstRound+uint64(roundRange)) } -} \ No newline at end of file +} diff --git a/network/rounds/retrieve.go b/network/rounds/retrieve.go index 00f6675375243ef4025ad86f87751b578f185c1c..7151dd92de4d110e8a79f514e0dd4a1190e065a9 100644 --- a/network/rounds/retrieve.go +++ b/network/rounds/retrieve.go @@ -49,7 +49,7 @@ func (m *Manager) processMessageRetrieval(comms messageRetrievalComms, case rl := <-m.lookupRoundMessages: ri := rl.roundInfo jww.DEBUG.Printf("Checking for messages in round %d", ri.ID) - if !m.params.RealtimeOnly{ + if !m.params.RealtimeOnly { err := m.Session.UncheckedRounds().AddRound(id.Round(ri.ID), ri, rl.identity.Source, rl.identity.EphId) if err != nil { @@ -57,7 +57,6 @@ func (m *Manager) processMessageRetrieval(comms messageRetrievalComms, } } - // Convert gateways in round to proper ID format gwIds := make([]*id.ID, len(ri.Topology)) for i, idBytes := range ri.Topology { @@ -132,7 +131,7 @@ func (m *Manager) processMessageRetrieval(comms messageRetrievalComms, m.messageBundles <- bundle jww.DEBUG.Printf("Removing round %d from unchecked store", ri.ID) - if !m.params.RealtimeOnly{ + if !m.params.RealtimeOnly { err = m.Session.UncheckedRounds().Remove(id.Round(ri.ID), rl.identity.Source, rl.identity.EphId) if err != nil { jww.ERROR.Printf("Could not remove round %d "+ @@ -140,7 +139,6 @@ func (m *Manager) processMessageRetrieval(comms messageRetrievalComms, } } - } } @@ -196,14 +194,13 @@ func (m *Manager) getMessagesFromGateway(roundID id.Round, " in round %d. This happening every once in a while is normal,"+ " but can be indicative of a problem if it is consistent", m.TransmissionID, roundID) - if m.params.RealtimeOnly{ + if m.params.RealtimeOnly { err = m.Session.UncheckedRounds().Remove(roundID, identity.Source, identity.EphId) if err != nil { jww.ERROR.Printf("Failed to remove round %d: %+v", roundID, err) } } - return message.Bundle{}, nil } diff --git a/network/rounds/utils_test.go b/network/rounds/utils_test.go index 8779a68acdf4f0c929186ddb59b6a0ad26bc3f05..ea24534930b93b345ef2bff86de0f29f6663507b 100644 --- a/network/rounds/utils_test.go +++ b/network/rounds/utils_test.go @@ -62,10 +62,10 @@ func (mmrc *mockMessageRetrievalComms) RemoveHost(hid *id.ID) { } func (mmrc *mockMessageRetrievalComms) GetHost(hostId *id.ID) (*connect.Host, bool) { - h, _ := connect.NewHost(hostId, "0.0.0.0", []byte(""), connect.HostParams{ - MaxRetries: 0, - AuthEnabled: false, - }) + p := connect.GetDefaultHostParams() + p.MaxRetries = 0 + p.AuthEnabled = false + h, _ := connect.NewHost(hostId, "0.0.0.0", []byte(""), p) return h, true } diff --git a/single/responseMessage.go b/single/responseMessage.go index 72d3bbdb666f4d6153ed59cf20caa9e64b49d7fd..191038dcb7665b17578268c85623af858b1601fb 100644 --- a/single/responseMessage.go +++ b/single/responseMessage.go @@ -14,11 +14,11 @@ import ( ) const ( - partNumLen = 1 - maxPartsLen = 1 - responseMinSize = receptionMessageVersionLen + partNumLen + maxPartsLen + sizeSize + partNumLen = 1 + maxPartsLen = 1 + responseMinSize = receptionMessageVersionLen + partNumLen + maxPartsLen + sizeSize receptionMessageVersion = 0 - receptionMessageVersionLen = 1 + receptionMessageVersionLen = 1 ) /* @@ -60,7 +60,7 @@ func mapResponseMessagePart(data []byte) responseMessagePart { return responseMessagePart{ data: data, version: data[:receptionMessageVersionLen], - partNum: data[receptionMessageVersionLen:receptionMessageVersionLen+partNumLen], + partNum: data[receptionMessageVersionLen : receptionMessageVersionLen+partNumLen], maxParts: data[receptionMessageVersionLen+partNumLen : receptionMessageVersionLen+maxPartsLen+partNumLen], size: data[receptionMessageVersionLen+maxPartsLen+partNumLen : responseMinSize], contents: data[responseMinSize:], diff --git a/single/responseMessage_test.go b/single/responseMessage_test.go index b97aa27b544b03779bc28c32e6a9232722fd815f..f9a78f068fb90021f1a7a9f9571ca02d244b91d1 100644 --- a/single/responseMessage_test.go +++ b/single/responseMessage_test.go @@ -21,7 +21,7 @@ func Test_newResponseMessagePart(t *testing.T) { payloadSize := prng.Intn(2000) expected := responseMessagePart{ data: make([]byte, payloadSize), - version: make([]byte, receptionMessageVersionLen), + version: make([]byte, receptionMessageVersionLen), partNum: make([]byte, partNumLen), maxParts: make([]byte, maxPartsLen), size: make([]byte, sizeSize), diff --git a/single/response_test.go b/single/response_test.go index 2a6c74521b593b490f3ee53f92aeff488f6bba7f..fcacf19a51bb2c15bcdb961b53665e67248837da 100644 --- a/single/response_test.go +++ b/single/response_test.go @@ -23,7 +23,7 @@ import ( func TestManager_GetMaxResponsePayloadSize(t *testing.T) { m := newTestManager(0, false, t) cmixPrimeSize := m.store.Cmix().GetGroup().GetP().ByteLen() - expectedSize := 2*cmixPrimeSize - format.KeyFPLen - format.MacLen - format.RecipientIDLen - responseMinSize-1 + expectedSize := 2*cmixPrimeSize - format.KeyFPLen - format.MacLen - format.RecipientIDLen - responseMinSize - 1 testSize := m.GetMaxResponsePayloadSize() if expectedSize != testSize { diff --git a/single/transmitMessage.go b/single/transmitMessage.go index 6676a3ae5d754efa6bf1434e9ccb263214f23bb6..17ce73acd2a6a5183c5c173f31eb23bd950f41c7 100644 --- a/single/transmitMessage.go +++ b/single/transmitMessage.go @@ -61,7 +61,7 @@ func mapTransmitMessage(data []byte, pubKeySize int) transmitMessage { return transmitMessage{ data: data, version: data[:transmitMessageVersionSize], - pubKey: data[transmitMessageVersionSize:transmitMessageVersionSize+pubKeySize], + pubKey: data[transmitMessageVersionSize : transmitMessageVersionSize+pubKeySize], payload: data[transmitMessageVersionSize+pubKeySize:], } } diff --git a/single/transmitMessage_test.go b/single/transmitMessage_test.go index 5125526ab47e3d98e7f9e4c81306492f129a1772..b6c62d25b71f507d54d1f69a70c0a35e932af628 100644 --- a/single/transmitMessage_test.go +++ b/single/transmitMessage_test.go @@ -140,7 +140,7 @@ func TestTransmitMessage_SetPayload_GetPayload_GetPayloadSize(t *testing.T) { prng := rand.New(rand.NewSource(42)) externalPayloadSize := prng.Intn(2000) pubKeySize := prng.Intn(externalPayloadSize) - payloadSize := externalPayloadSize - pubKeySize-transmitMessageVersionSize + payloadSize := externalPayloadSize - pubKeySize - transmitMessageVersionSize payload := make([]byte, payloadSize) prng.Read(payload) m := newTransmitMessage(externalPayloadSize, pubKeySize) @@ -153,7 +153,6 @@ func TestTransmitMessage_SetPayload_GetPayload_GetPayloadSize(t *testing.T) { "\nexpected: %+v\nreceived: %+v", payload, testPayload) } - if payloadSize != m.GetPayloadSize() { t.Errorf("GetContentsSize() returned incorrect content size."+ "\nexpected: %d\nreceived: %d", payloadSize, m.GetPayloadSize()) diff --git a/storage/cmix/store.go b/storage/cmix/store.go index 641261c9d85fb0f2718058c420ecb797cd00aaea..5947900089ea3ec047c6ce676eb27264cc20dca9 100644 --- a/storage/cmix/store.go +++ b/storage/cmix/store.go @@ -23,17 +23,17 @@ import ( const prefix = "cmix" const currentStoreVersion = 0 const ( - storeKey = "KeyStore" - grpKey = "GroupKey" + storeKey = "KeyStore" + grpKey = "GroupKey" ) type Store struct { - nodes map[id.ID]*key - validUntil uint64 - keyId []byte - grp *cyclic.Group - kv *versioned.KV - mux sync.RWMutex + nodes map[id.ID]*key + validUntil uint64 + keyId []byte + grp *cyclic.Group + kv *versioned.KV + mux sync.RWMutex } // NewStore returns a new cMix storage object. @@ -42,9 +42,9 @@ func NewStore(grp *cyclic.Group, kv *versioned.KV) (*Store, error) { kv = kv.Prefix(prefix) s := &Store{ - nodes: make(map[id.ID]*key), - grp: grp, - kv: kv, + nodes: make(map[id.ID]*key), + grp: grp, + kv: kv, } err := utility.StoreGroup(kv, grp, grpKey) diff --git a/storage/fileTransfer/fileMessage.go b/storage/fileTransfer/fileMessage.go index 03269326bbeab891e4678c888994fc35ea1026a6..90fd6eb30345f2aa5eda31d313058aaadd437ee6 100644 --- a/storage/fileTransfer/fileMessage.go +++ b/storage/fileTransfer/fileMessage.go @@ -78,7 +78,7 @@ func UnmarshalPartMessage(b []byte) (PartMessage, error) { // Marshal returns the byte representation of the PartMessage. func (m PartMessage) Marshal() []byte { b := make([]byte, len(m.data)) - copy(b,m.data) + copy(b, m.data) return b } @@ -97,7 +97,7 @@ func (m PartMessage) SetPartNum(num uint16) { // GetPart returns the file part data from the message. func (m PartMessage) GetPart() []byte { b := make([]byte, len(m.part)) - copy(b,m.part) + copy(b, m.part) return b } diff --git a/storage/fileTransfer/receiveTransfer_test.go b/storage/fileTransfer/receiveTransfer_test.go index 281bb373822ee0642a6c02c07c814d402cd3d174..0edd3cfe63112c519c45046a009ecce868c3cde3 100644 --- a/storage/fileTransfer/receiveTransfer_test.go +++ b/storage/fileTransfer/receiveTransfer_test.go @@ -557,7 +557,7 @@ func TestReceivedTransfer_AddPart(t *testing.T) { cmixMsg.SetMac(mac) // Add encrypted part - complete, err := rt.AddPart(cmixMsg,fpNum) + complete, err := rt.AddPart(cmixMsg, fpNum) if err != nil { t.Errorf("AddPart returned an error: %+v", err) } @@ -1156,4 +1156,4 @@ func newEmptyReceivedTransfer(numParts, numFps uint16, kv *versioned.KV, } return tid, rt, fileData -} \ No newline at end of file +} diff --git a/storage/fileTransfer/sentTransfer.go b/storage/fileTransfer/sentTransfer.go index cdffaaeca6807c4025b9bf90925130e6be9cbcb5..115eee1de2d8a95ef302edec84d6271e1ddce4ce 100644 --- a/storage/fileTransfer/sentTransfer.go +++ b/storage/fileTransfer/sentTransfer.go @@ -447,7 +447,7 @@ func (st *SentTransfer) GetEncryptedPart(partNum uint16, contentsSize int) (encP errors.Errorf(noPartNumErr, partNum) } - if err = partMsg.SetPart(part); err != nil{ + if err = partMsg.SetPart(part); err != nil { return nil, nil, format.Fingerprint{}, err } diff --git a/storage/fileTransfer/sentTransfer_test.go b/storage/fileTransfer/sentTransfer_test.go index 033992bf84b42aa4bf54431ac50e3c3c0b132998..ea4302b47a857125522b97be4cc13c4503e2d39c 100644 --- a/storage/fileTransfer/sentTransfer_test.go +++ b/storage/fileTransfer/sentTransfer_test.go @@ -850,9 +850,9 @@ func TestSentTransfer_GetEncryptedPart(t *testing.T) { "\nexpected: %+v\nreceived: %+v", partNum, i, expectedPart, partMsg.GetPart()) } - if partMsg.GetPartNum()!=i % st.numParts{ - t.Errorf("Number of part did not match, expected: %d, " + - "received: %d", i % st.numParts, partMsg.GetPartNum()) + if partMsg.GetPartNum() != i%st.numParts { + t.Errorf("Number of part did not match, expected: %d, "+ + "received: %d", i%st.numParts, partMsg.GetPartNum()) } } } diff --git a/storage/utility/messageBuffer.go b/storage/utility/messageBuffer.go index d9c7ad749a3b1011c976fe97387dac281aec7086..f9891edce39ff284406bd1878df8a627f3a7589b 100644 --- a/storage/utility/messageBuffer.go +++ b/storage/utility/messageBuffer.go @@ -270,7 +270,7 @@ func (mb *MessageBuffer) Next() (interface{}, bool) { // Retrieve the message for storage m, err = mb.handler.LoadMessage(mb.kv, makeStoredMessageKey(mb.key, h)) if err != nil { - m=nil + m = nil jww.ERROR.Printf("Failed to load message %s from store, "+ "this may happen on occasion due to replays to increase "+ "reliability: %v", h, err) @@ -292,7 +292,7 @@ func next(msgMap map[MessageHash]struct{}) MessageHash { func (mb *MessageBuffer) Succeeded(m interface{}) { h := mb.handler.HashMessage(m) jww.TRACE.Printf("Critical Messages Succeeded(%s)", - base64.StdEncoding.EncodeToString((h[:]))) + base64.StdEncoding.EncodeToString(h[:])) mb.mux.Lock() defer mb.mux.Unlock() diff --git a/storage/utility/meteredCmixMessageBuffer.go b/storage/utility/meteredCmixMessageBuffer.go index 9060dbb7ea2424f87879032371ee25944ff64851..b0b425800738fba7af41ae7339bafdf6ffa1cd6d 100644 --- a/storage/utility/meteredCmixMessageBuffer.go +++ b/storage/utility/meteredCmixMessageBuffer.go @@ -114,7 +114,7 @@ func LoadMeteredCmixMessageBuffer(kv *versioned.KV, key string) (*MeteredCmixMes } func (mcmb *MeteredCmixMessageBuffer) Add(m format.Message) { - if m.GetPrimeByteLen()==0{ + if m.GetPrimeByteLen() == 0 { jww.FATAL.Panicf("Cannot handle a metered " + "cmix message with a length of 0") } diff --git a/xxmutils/restoreContacts.go b/xxmutils/restoreContacts.go index 6bced7e15e9835024e93e2f327b8c7cc8f48b14d..b64947418cb401ea797c99cc8c5935a60335bfb9 100644 --- a/xxmutils/restoreContacts.go +++ b/xxmutils/restoreContacts.go @@ -87,9 +87,9 @@ func RestoreContactsFromBackup(backupPartnerIDs []byte, client *api.Client, failCh := make(chan failure, chanSize) // Start routines for processing - lcWg := sync.WaitGroup{} + lcWg := &sync.WaitGroup{} lcWg.Add(numRoutines) - rsWg := sync.WaitGroup{} + rsWg := &sync.WaitGroup{} rsWg.Add(numRoutines) for i := 0; i < numRoutines; i++ { go LookupContacts(lookupCh, foundCh, failCh, udManager, lcWg) @@ -149,13 +149,18 @@ func RestoreContactsFromBackup(backupPartnerIDs []byte, client *api.Client, } // Cleanup + // lookupCh -> foundCh -> resetContactCh -> restoredCh close(lookupCh) - close(resetContactCh) - close(failCh) // Now wait for subroutines to close before closing their output chans lcWg.Wait() + // Close input to reset chan after lookup is done to avoid writes after + // close close(foundCh) + close(resetContactCh) rsWg.Wait() + // failCh is closed after exit of the threads to avoid writes after + // close + close(failCh) close(restoredCh) failWg.Wait() @@ -168,7 +173,7 @@ func RestoreContactsFromBackup(backupPartnerIDs []byte, client *api.Client, // should be treated as internal functions specific to the phone apps. func LookupContacts(in chan *id.ID, out chan *contact.Contact, failCh chan failure, udManager *ud.Manager, - wg sync.WaitGroup) { + wg *sync.WaitGroup) { defer wg.Done() // Start looking up contacts with user discovery and feed this // contacts channel. @@ -185,8 +190,6 @@ func LookupContacts(in chan *id.ID, out chan *contact.Contact, continue } jww.WARN.Printf("could not lookup %s: %v", lookupID, err) - // Retry later - in <- lookupID } } @@ -196,7 +199,7 @@ func LookupContacts(in chan *id.ID, out chan *contact.Contact, // the mobile phone apps and are not intended to be part of the xxDK. It // should be treated as internal functions specific to the phone apps. func ResetSessions(in, out chan *contact.Contact, failCh chan failure, - client api.Client, wg sync.WaitGroup) { + client api.Client, wg *sync.WaitGroup) { defer wg.Done() me := client.GetUser().GetContact() msg := "Account reset from backup" @@ -207,9 +210,9 @@ func ResetSessions(in, out chan *contact.Contact, failCh chan failure, continue } // If an error, figure out if I should report or retry - // Note: Always retry here for now. + // Note: Always fail here for now. jww.WARN.Printf("could not reset %s: %v", c.ID, err) - in <- c + failCh <- failure{ID: c.ID, Err: err} } }