Tutorial: Build an API requester

This tutorial walks you through the process of creating a simple Golang program that can call Insolar MainNet API.

Note

You can also use the CLI tool of the same name (requester) to call the API.

Code examples in this tutorial are straightforward, statements are successive (without conditional expressions and cycles). This lets you focus on substance rather than code structure: how to use cryptographic functions provided by Golang packages to correctly sign API requests to MainNet.

You can learn to sign requests in two ways:

  • By creating a new private key (together with a new Insolar Wallet), or

  • By using an existing private key if you have already created your Wallet in the web interface.

Note

This tutorial uses the API endpoint of Insolar TestNet in code examples.

What you will build

You will build a program that creates a member on the Insolar network or uses an existing one and, as a member, transfers funds from its account to the account of another member.

What you will need

How to complete this tutorial

  • To start from scratch, go through the step-by-step instructions listed below and pay attention to comments in code examples.

  • To skip the basics, read (and copy-paste) a working requester code provided at the end of this tutorial.

Building the requester

In Insolar MainNet (and TestNet), all requests to contracts go through two phases:

Phase 1: A client requests a seed from a node. The seed is a unique piece of information (explained in more detail in the following sections).

Phase 2: The client forms and sends a contract request in the following way:

  1. Puts the seed value and a public key into a contract request body.

  2. Takes a hash of the body’s bytes.

  3. Signs the hash with a private key.

  4. Puts both the hash and signature into corresponding headers.

  5. Sends the request to the node that provided the seed in the previous phase.

To automate this process, you can build a simple requester program by completing the following steps:

  1. Prepare: install the necessary tools, import the necessary packages, and initialize an HTTP client.

  2. Declare structures that correspond to request bodies described in the Insolar API specification.

  3. Create a seed getter function. The getter is reused to put a new seed into every contract request.

  4. Create a sender function that signs and sends contract requests given a private key.

  5. Generate a new private key or use an existing one.

  6. Form and send a transfer request: get a new seed and call the sender function.

  7. Test the requester against Insolar TestNet.

All the above steps are detailed in sections below.

Step 1: Prepare

To build the requester, install, import, and set up the following:

  1. Install a copy of the standard crypto library with the secp256k1 elliptic curve implementation provided by Insolar:

    go get -t github.com/insolar/x-crypto/ecdsa/...
    
  2. In a new Main.go file, import the packages your requester will use (or skip this step and let your IDE do it for you along the way). For example:

     1package main
     2
     3import (
     4  // You will need:
     5  // - Basic Golang functionality.
     6  "bytes"
     7  "fmt"
     8  "golang.org/x/net/publicsuffix"
     9  "io/ioutil"
    10  "log"
    11  "strconv"
    12  // - HTTP client and a cookiejar.
    13  "net/http"
    14  "net/http/cookiejar"
    15  // - Big numbers to store signatures.
    16  "math/big"
    17  // - Basic cryptography.
    18  "crypto/rand"
    19  "crypto/sha256"
    20  // - Basic encoding capabilities.
    21  "encoding/asn1"
    22  "encoding/base64"
    23  "encoding/json"
    24  "encoding/pem"
    25  // - A copy of the standard crypto library with
    26  //   the ECDSA secp256k1 curve implementation.
    27  xecdsa "github.com/insolar/x-crypto/ecdsa"
    28  xelliptic "github.com/insolar/x-crypto/elliptic"
    29  "github.com/insolar/x-crypto/x509"
    30)
    
  3. Declare, set, and initialize the following:

    1. Insolar supports ECDSA-signed requests. Since an ECDSA signature in Golang consists of two big integers, declare a single structure to contain it.

    2. Set the API endpoint URL to that of TestNet.

    3. Create and initialize an HTTP client for connection reuse and store a cookiejar inside.

    4. Create a variable for the JSON RPC 2.0 request identifier. The identifier is to be incremented for every request and each corresponding response will contain it.

    31// Declare a structure to contain the ECDSA signature.
    32type ecdsaSignature struct {
    33  R, S *big.Int
    34}
    35
    36// Set the endpoint URL to that of TestNet.
    37const (
    38  TestNetURL = "https://wallet-api.testnet.insolar.io/api/rpc"
    39)
    40
    41// Create and initialize an HTTP client for connection reuse
    42// and put a cookiejar into it.
    43var client *http.Client
    44var jar cookiejar.Jar
    45func init() {
    46  // All users of cookiejar should import "golang.org/x/net/publicsuffix"
    47  jar, err := cookiejar.New(&cookiejar.Options{
    48    PublicSuffixList: publicsuffix.List})
    49  if err != nil {
    50    log.Fatal(err)
    51  }
    52  client = &http.Client{
    53    Jar: jar,
    54  }
    55}
    56
    57// Create a variable for the JSON RPC 2.0 request identifier.
    58var id int = 1
    59// The identifier is incremented in every request
    60// and each corresponding response contains it.
    

With that, everything your requester requires is set up.

Next, declare request structures in accordance with the Insolar API specification.

Step 2: Declare request structures

To call the MainNet (or TestNet) API, you need structures for three requests: seed getter, member creation, and transfer.

All the requests have the same base structure in accordance with the JSON RPC 2.0 specification. For example:

{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "contract.call"
  "params": { ... }
}

Where "params" is an optional object that may contain parameters of a particular method.

Define the base structure and nest more structures for all the required parameters. For example:

 61// Continue in the Main.go file...
 62
 63// Declare a base structure to form requests to Insolar API
 64// in accordance with the specification.
 65type requestBody struct {
 66  JSONRPC        string         `json:"jsonrpc"`
 67  ID             int            `json:"id"`
 68  Method         string         `json:"method"`
 69}
 70
 71type requestBodyWithParams struct {
 72  JSONRPC        string         `json:"jsonrpc"`
 73  ID             int            `json:"id"`
 74  Method         string         `json:"method"`
 75  // Params is a structure that depends on a particular method.
 76  Params         interface{}    `json:"params"`
 77}
 78
 79// Insolar MainNet defines params of a contract request as follows.
 80type params struct {
 81  Seed            string       `json:"seed"`
 82  CallSite        string       `json:"callSite"`
 83  // CallParams is a structure that depends on a particular method.
 84  CallParams      interface{}  `json:"callParams"`
 85  PublicKey       string       `json:"publicKey"`
 86}
 87
 88// The transfer request has a reference in params.
 89type paramsWithReference struct {
 90  params
 91  Reference       string  `json:"reference"`
 92}
 93
 94// The member.create request has no callParams,
 95// so here goes an empty structure.
 96type memberCreateCallParams struct {}
 97
 98// The transfer request sends an amount of funds to
 99// a member identified by a reference.
100type transferCallParams struct {
101  Amount            string    `json:"amount"`
102  ToMemberReference string    `json:"toMemberReference"`
103}

Now that the requester has all the requests structures it is supposed to use, the next step is to create the following functions:

  1. A seed getter to retrieve a new seed for each contract request.

  2. A sender function that signs and sends contract requests.

Step 3: Create a seed getter

Each signed request to Insolar API has to contain a seed in its body. Seed is a unique piece of information generated by a node that:

  • Has a short lifespan.

  • Expires upon first use.

  • Protects from request duplicates.

Upon receiving a contract request, any node checks if it was the one that generated the seed and if the seed is still alive. So, each contract request with a seed must be sent to the node you requested the seed from.

Tip

To make sure that the contract request is routed to the correct node, retrieve all the cookies with routing information from the node’s response and store them in the HTTP client as described in the preparation step.

To form contract requests, create a seed getter function you can reuse.

The seed getter does the following:

  1. Forms a node.getSeed request body in JSON format.

  2. Creates an HTTP request with the body and a Content-Type (application/json) HTTP header.

  3. Sends the request and receives a response.

  4. Retrieves the seed from the response and returns it.

For example:

104// Continue in the Main.go file...
105
106// Create a function to get a new seed for each signed request.
107func getNewSeed() (string) {
108  // Form a request body for getSeed.
109  getSeedReq := requestBody{
110    JSONRPC: "2.0",
111    Method:  "node.getSeed",
112    ID:      id,
113  }
114  // Increment the id for future requests.
115  id++
116
117  // Marshal the payload into JSON.
118  jsonSeedReq, err := json.Marshal(getSeedReq)
119  if err != nil {
120    log.Fatalln(err)
121  }
122
123  // Create a new HTTP request.
124  seedReq, err := http.NewRequest("POST", TestNetURL,
125    bytes.NewBuffer(jsonSeedReq))
126  if err != nil {
127    log.Fatalln(err)
128  }
129  seedReq.Header.Set("Content-Type", "application/json")
130
131  // Send the request.
132  seedResponse, err := client.Do(seedReq)
133  if err != nil {
134    log.Fatalln(err)
135  }
136  defer seedReq.Body.Close()
137
138  // Receive the response body.
139  seedRespBody, err := ioutil.ReadAll(seedResponse.Body)
140  if err != nil {
141    log.Fatalln(err)
142  }
143
144  // Unmarshal the response.
145  var newSeed map[string]interface{}
146  err = json.Unmarshal(seedRespBody, &newSeed)
147  if err != nil {
148    log.Fatalln(err)
149  }
150
151  // (Optional) Print the request and its response.
152  print := "POST to " + TestNetURL +
153    "\nPayload: " + string(jsonSeedReq) +
154    "\nResponse status code: " +  strconv.Itoa(seedResponse.StatusCode) +
155    "\nResponse: " + string(seedRespBody) + "\n"
156  fmt.Println(print)
157
158  // Retrieve and return the new seed.
159  return newSeed["result"].(map[string]interface{})["seed"].(string)
160}

Now, every getNewSeed() call returns a living seed that can be put into the body of a contract request.

The next step is to create a sender function that signs and sends such requests.

Step 4: Create a sender function

The sender function does the following:

  1. Takes a request body and an ECDSA private key as arguments.

  2. Forms an HTTP request with the body and the following HTTP headers:

    1. Content-Typeapplication/json.

    2. Digest that contains a Base64 string with an SHA-256 hash of the body’s bytes.

    3. Signature that contains a Base64 string with an ECDSA signature (in ASN.1 DER format) of the hash’s bytes.

  3. Sends the request.

  4. Retrieves the response and returns it as a JSON object.

For example:

Tip

In Golang, the ECDSA signature consists of two big integers. To convert the signature into the ASN.1 DER format, put it into the ecdsaSignature structure.

161// Continue in the Main.go file...
162
163// Create a function to send signed requests.
164func sendSignedRequest(payload requestBodyWithParams,
165  privateKey *ecdsa.PrivateKey) map[string]interface{} {
166
167  // Marshal the payload into JSON.
168  jsonPayload, err := json.Marshal(payload)
169  if err != nil {
170    log.Fatalln(err)
171  }
172
173  // Take a SHA-256 hash of the payload's bytes.
174  hash := sha256.Sum256(jsonPayload)
175
176  // Sign the hash with the private key.
177  r, s, err := ecdsa.Sign(rand.Reader, privateKey, hash[:])
178  if err != nil {
179    log.Fatalln(err)
180  }
181
182  // Convert the signature into ASN.1 DER format.
183  sig := ecdsaSignature{
184    R: r,
185    S: s,
186  }
187  signature, err := asn1.Marshal(sig)
188  if err != nil {
189    log.Fatalln(err)
190  }
191
192  // Encode both hash and signature to a Base64 string.
193  hash64 := base64.StdEncoding.EncodeToString(hash[:])
194  signature64 := base64.StdEncoding.EncodeToString(signature)
195
196  // Create a new request and set its headers.
197  request, err := http.NewRequest("POST", TestNetURL,
198    bytes.NewBuffer(jsonPayload))
199  if err != nil {
200    log.Fatalln(err)
201  }
202  request.Header.Set("Content-Type", "application/json")
203
204  // Put the hash string into the HTTP Digest header.
205  request.Header.Set("Digest", "SHA-256="+hash64)
206
207  // Put the signature string into the HTTP Signature header.
208  request.Header.Set("Signature", "keyId=\"public-key\", " +
209     "algorithm=\"ecdsa\", headers=\"digest\", signature="+signature64)
210
211  // Send the signed request.
212  response, err := client.Do(request)
213  if err != nil {
214    log.Fatalln(err)
215  }
216  defer response.Body.Close()
217
218  // Receive the response body.
219  responseBody, err := ioutil.ReadAll(response.Body)
220  if err != nil {
221    log.Fatalln(err)
222  }
223
224  // Unmarshal it into a JSON object.
225  var JSONObject map[string]interface{}
226  err = json.Unmarshal(responseBody, &JSONObject)
227  if err != nil {
228    log.Fatalln(err)
229  }
230
231  // (Optional) Print the request and its response.
232  print := "POST to " + TestNetURL +
233    "\nPayload: " + string(jsonPayload) +
234    "\nResponse status code: " + strconv.Itoa(response.StatusCode) +
235    "\nResponse: " + string(responseBody) + "\n"
236  fmt.Println(print)
237
238  // Return the JSON object.
239  return JSONObject
240}

Now, every sendSignedRequest(payload, privateKey) call returns the result of a contract method execution.

With the seed getter and sender functions, you have everything you need to send a contract request. The next step is to:

  • Generate a key pair and create a member using a special contract request, or

  • Use an existing member account by retrieving the corresponding private key from the Insolar Wallet’s web interface and converting the key to PEM format.

Step 5: Generate a new key pair or use an existing one

The body of each request that calls a contract method must be hashed by a SHA256 algorithm. Each hash must be signed by a private key generated by a p256k1 elliptic curve.

Depending on whether or not you already have an Insolar Wallet, choose one of the following:

To create a member, send the corresponding member creation request—a signed request to a contract method that does the following:

  • Creates a new member and corresponding account objects.

  • Returns a reference to the member—address in the Insolar network.

  • Binds a given public key to the member.

Insolar uses this public key to identify a member and check the signature generated by the paired private key.

Warning

You will not be able to access your member object without the private key and, as such, transfer funds.

First, take care of the keys by following these steps:

  1. Generate a key pair using the elliptic curve and convert both keys to PEM format.

  2. Export the private key into a file.

  3. Save the file to a secure place.

Next, form and sigh the member creation request:

  1. Call the getNewSeed() function and put a new seed into a variable.

  2. Form the member.create request body with the seed and the generated public key.

  3. Call the sendSignedRequest() function, pass it the body and the private key, and receive a member reference in response.

  4. Put the reference into a variable (the transfer request in the next step requires it).

For example:

Tip

To encode the key to PEM format, first, convert it to ASN.1 DER using the x509 library.

241// Continue in the Main.go file...
242
243// Create the main function to form and send signed requests.
244func main() {
245
246  // Generate a key pair.
247  privateKey := new(xecdsa.PrivateKey)
248  privateKey, err := xecdsa.GenerateKey(xelliptic.P256(), rand.Reader)
249  var publicKey xecdsa.PublicKey
250  publicKey = privateKey.PublicKey
251
252  // Convert both private and public keys into PEM format.
253  x509PublicKey, err := x509.MarshalPKIXPublicKey(&publicKey)
254  if err != nil {
255    log.Fatalln(err)
256  }
257  pemPublicKey := pem.EncodeToMemory(&pem.Block{Type: "PUBLIC KEY",
258     Bytes: x509PublicKey})
259
260  x509PrivateKey, err := x509.MarshalECPrivateKey(privateKey)
261  if err != nil {
262    log.Fatalln(err)
263  }
264  pemPrivateKey := pem.EncodeToMemory(&pem.Block{Type: "PRIVATE KEY",
265     Bytes: x509PrivateKey})
266
267  // The private key is required to sign requests.
268  // Make sure to put it into a file to save it to a secure place later.
269  file, err := os.Create("private.pem")
270  if err != nil {
271    fmt.Println(err)
272    return
273  }
274  file.WriteString(string(pemPrivateKey))
275  file.Close()
276
277  // Get a seed to form the request.
278  seed := getNewSeed()
279  // Form a request body for member.create.
280  createMemberReq := requestBodyWithParams{
281    JSONRPC: "2.0",
282    Method:  "contract.call",
283    ID:      id,
284    Params:params {
285      Seed: seed,
286      CallSite: "member.create",
287      CallParams:memberCreateCallParams {},
288      PublicKey: string(pemPublicKey)},
289  }
290  // Increment the JSON RPC 2.0 request identifier for future requests.
291  id++
292
293  // Send the signed member.create request.
294  newMember := sendSignedRequest(createMemberReq, privateKey)
295
296  // Put the reference to your new member into a variable
297  // to easily form transfer requests.
298  memberReference := newMember["result"].(
299  map[string]interface{})["callResult"].(
300  map[string]interface{})["reference"].(string)
301  fmt.Println("Member reference is " + memberReference)
302
303  // The main function is to be continued...

Now that you have your member reference and key pair, you can transfer funds to other members.

Step 6: Form and send a transfer request

The transfer request is a signed request to a contract method that transfers an amount of funds to another member.

To transfer funds, follow these steps:

  1. In the web interface, send some funds to your XNS address or member reference returned by the member creation request.

  2. Acquire a recipient reference—reference to an existing member to transfer the funds to.

  3. Call the getNewSeed() function and put a new seed into a variable.

  4. Form a member.transfer request body with the following values:

    • A new seed

    • An amount of funds to transfer

    • A recipient reference

    • Your reference (XNS address)—for identification

    • Your public key—to check the signature

  5. Call the sendSignedRequest() function and pass it the body and the private key.

The transfer request responds with a fee value.

For example:

Attention

In the highlighted line, replace the insolar:YYY... placeholder with the reference to an existing recipient member.

304// Continue in the main() function...
305
306// Get a new seed to form a transfer request.
307seed = getNewSeed()
308
309// Form a request body for the transfer request.
310transferReq := requestBodyWithParams{
311  JSONRPC: "2.0",
312  Method:  "contract.call",
313  ID:      id,
314  Params:paramsWithReference{ params:params{
315    Seed: seed,
316    CallSite: "member.transfer",
317    CallParams:transferCallParams {
318      Amount: "100",
319      ToMemberReference: "insolar:YYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY",
320      },
321    PublicKey: string(pemPublicKey),
322    },
323    Reference: string(memberReference),
324  },
325}
326// Increment the id for future requests.
327id++
328
329// Send the signed transfer request.
330newTransfer := sendSignedRequest(transferReq, privateKey)
331fee := newTransfer["result"].(
332  map[string]interface{})["callResult"].(
333  map[string]interface{})["fee"].(string)
334
335// (Optional) Print out the fee.
336fmt.Println("Fee is " + fee)
337
338// Remember to close the main function.
339}

With that, the requester, as a member, can send funds to other members of the Insolar network.

Step 7: Test the requester

To test the requester, do the following:

  1. Make sure the endpoint URL is set to that of TestNet.

  2. Run the requester:

    go run Main.go
    

Summary

Congratulations! You have just developed a requester capable of forming signed contract requests to Insolar MainNet API.

Build upon it:

  1. Create structures for other contract requests.

  2. Export the getter and sender functions to use them in other packages.

Full requester code examples

Below are the full requester code examples in Golang. Click the panels to expand and click again to hide.