Understanding how a Solana transaction is constructed is crucial for building Solana applications like trading bots, smart contracts, DeFi applications, etc. It's very useful for people who are new to Solana or blockchain in general. In this article, we will learn how to create a Solana transaction without using any client libraries to gain a deeper understanding of how Solana transactions work.

Core concepts

Before continuing, I highly recommend reading the official Solana documentation about Solana Account Model and Transactions and Instructions. They provide a good foundation for understanding Solana account and transaction models.

I'll quickly go over some concepts here:

Account

1. Arrow
   The way Solana stores data resembles a key-value store where each account has a unique address and acts as an entry in the store.
2. Arrow
   An account can be either a program (smart contract) account or a user (wallet) account.
3. Arrow
   Every account has a program "owner". Only the program that owns an account can modify its data or deduct its lamport balance. However, anyone can increase the balance.
4. Arrow
   In a transaction, an account is a signer if it's required to sign the transaction. Additionally, an account is writable if its data/state can be modified by the transaction.

Transaction

1. Arrow
   Sending a transaction is the way to interact (sending or receiving SOL, calling programs, etc.) with the Solana blockchain.
2. Arrow
   A transaction consists of a list of instructions where each instruction represents an operation to be executed. Each instruction specifies the program to call, the accounts to pass to the program, and the data to send to the program.
3. Arrow
   Instructions are executed sequentially and atomically. If any instruction fails, the transaction is rolled back.
4. Arrow
   The maximum size of a transaction is 1232 bytes.

You can think of a Solana transaction as when you make an order at a restaurant. The order is a transaction, and each dish is an instruction. Or if you're familiar with web development, a transaction is like a POST request, except you can do multiple POST requests in one transaction.

Structure Of A Solana Transaction

It seems complicated, but don't worry; I'll explain each part. Feel free to skip to the next section if you already understand it.

Compact-Array Format

It contains 2 parts:

• Arrow
  The length of the array: encoded in a format called compact-u16.
• Arrow
  The items in the array: each item is sequentially listed after the encoded length.

Signatures

An array of signatures of the transaction. Each signature is 64 bytes long. Each signature is the result of signing the transaction (signatures + message) with the private key of the signer using the Ed25519 algorithm.

Message Header

It contains only 3 u8 fields:

1. Arrow
   The number of required signatures for the transaction: This determines how many signer accounts must provide valid signatures for the transaction to be processed.
2. Arrow
   The number of read-only (non-writable) and signer accounts: These accounts must sign the transaction to approve it, even though their data remains unchanged.
3. Arrow
   The number of read-only (non-writable) and non-signer accounts: These accounts are included for reference but do not require signatures or modification.

We can programmatically create the message header by iterating through the accounts required by each instruction and counting the number of signer and read-only accounts.

Compact Array Of Account Addresses

An array containing all the account addresses needed for the instructions within the transaction. Addresses in the array are ordered by the privileges for the accounts:

1. Arrow
   Accounts that are writable and signers
2. Arrow
   Accounts that are read-only and signers
3. Arrow
   Accounts that are writable and not signers
4. Arrow
   Accounts that are read-only and not signers

Just like the message header, we can programmatically create this array using the accounts required by the instructions.

Compact Array Of Instructions

Instruction

Each instruction must include the following information:

1. Arrow
   Program address: Specifies the program being invoked.
2. Arrow
   Accounts: Lists every account the instruction reads from or writes to, including other programs, using the AccountMeta struct.
3. Arrow
   Instruction Data: A byte array that specifies which instruction handler on the program to invoke as there can be multiple handlers in a program, plus any additional data required by the instruction handler (function arguments).

Compiled Instruction

The structure of an instruction is human-readable, but when we serialize it, it needs to be in a different format called CompiledInstruction. It contains a slightly different structure:

1. Arrow
   Program ID Index: This is represented as an u8 index pointing to an account address within the account addresses array.
2. Arrow
   Compact array of account address indexes: An array of u8 indexes pointing to the account addresses array for each account required by the instruction.
3. Arrow
   Compact array of opaque u8 data: A u8 byte array specific to the program invoked. This data specifies the instruction to invoke on the program along with any additional data that the instruction requires (such as function arguments).

Recent Blockhash

Finally, the final part of the transaction is the recent blockhash. This acts as a timestamp for the transaction and is used to prevent duplications and eliminate stale transactions.

The maximum age of a transaction's blockhash is 150 blocks (~1 minute assuming 400ms block times). If a transaction's blockhash is 150 blocks older than the latest blockhash, it is considered expired and the transaction will be rejected.

Implementation

Enough theory, let's implement a simple application that sends a small amount of SOL from one account to another. As I mentioned earlier, we won't use any client libraries, only the Solana JSON RPC API.

First, we need to figure out which program to call and what data it expects. In Solana, every "wallet" is owned by a program called System Program. This program provides basic functionalities like creating accounts, transferring SOL, etc. Regarding the data, the instruction handler we want to invoke is Transfer, its index is 2. Note that the type of the index is u32 so we need to encode 2 as a 32-bit integer in little-endian format.

The Transfer function requires 2 accounts: the sender and the receiver and an 64-bit unsigned integer representing the amount of SOL to transfer. The sender account must be a signer, and both accounts must be writable.

Now, let's define some types based on the structure of a Solana transaction:

1
// types.go
2
package main
3

4
type PublicKey [32]byte
5
type Signature [64]byte
6

7
type Transaction struct {
8
	Signatures []Signature
9
	Message    Message
10
}
11

12
type AccountMeta struct {
13
	PubKey     PublicKey
14
	IsSigner   bool
15
	IsWritable bool
16
}
17

18
type Instruction struct {
19
	ProgramID      PublicKey
20
	ProgramIDIndex uint8
21
	Accounts       []AccountMeta
22
	Data           []byte
23
}
24

25
type Message struct {
26
	Header          MessageHeader
27
	AccountKeys     []PublicKey
28
	RecentBlockhash PublicKey
29
	Instructions    []Instruction
30
}
31

32
type MessageHeader struct {
33
	NumRequiredSignatures       uint8
34
	NumReadonlySignedAccounts   uint8
35
	NumReadonlyUnsignedAccounts uint8
36
}

Next, we're going to create some helper functions:

1
// solana.go
2
package main
3

4
import (
5
	"bytes"
6
	"crypto"
7
	"crypto/ed25519"
8
	"crypto/rand"
9
	"encoding/binary"
10
	"encoding/json"
11
	"fmt"
12
	"io"
13
	"math"
14
	"net/http"
15
	"sort"
16

17
	"github.com/mr-tron/base58"
18
)
19

20
const (
21
	PublicKeyLength = 32
22
)
23

24
func MustPublicKeyFromBase58(in string) PublicKey {
25
	out, err := PublicKeyFromBase58(in)
26
	if err != nil {
27
		panic(err)
28
	}
29
	return out
30
}
31

32
func PublicKeyFromBase58(in string) (out PublicKey, err error) {
33
	val, err := base58.Decode(in)
34
	if err != nil {
35
		return out, fmt.Errorf("failed to decode base58: %w", err)
36
	}
37

38
	if len(val) != PublicKeyLength {
39
		return out, fmt.Errorf("invalid public key length: got %d, want %d", len(val), PublicKeyLength)
40
	}
41

42
	copy(out[:], val)
43
	return
44
}
45

46

47
// CreateTransferInstruction creates a transfer instruction
48
// https://docs.rs/solana-sdk/latest/solana_sdk/system_instruction/enum.SystemInstruction.html#variant.Transfer
49
func CreateTransferInstruction(sender, receiver PublicKey, lamports uint64) Instruction {
50
	systemProgramID := MustPublicKeyFromBase58("11111111111111111111111111111111")
51

52
	senderMeta := AccountMeta{
53
		PubKey:     sender,
54
		IsSigner:   true,
55
		IsWritable: true,
56
	}
57
	receiverMeta := AccountMeta{
58
		PubKey:     receiver,
59
		IsSigner:   false,
60
		IsWritable: true,
61
	}
62

63
	// Transfer instruction data
64
	// The first part is the instruction handler index, system program uses a 4-byte (u32) index, ref: https://github.com/solana-program/system/blob/5363ad3/clients/js/src/generated/programs/system.ts#L64-L66
65
	// The next 8 bytes (u64) are the amount in lamports (little-endian)
66
	data := make([]byte, 12)
67
	binary.LittleEndian.PutUint32(data[:4], 2)
68
	binary.LittleEndian.PutUint64(data[4:], lamports)
69

70
	return Instruction{
71
		ProgramID: systemProgramID,
72
		Accounts:  []AccountMeta{senderMeta, receiverMeta},
73
		Data:      data,
74
	}
75
}
76

77

78
func NewTransaction(instructions []Instruction, recentBlockhash [32]byte, feePayer PublicKey) (*Transaction, error) {
79
	message := Message{
80
		RecentBlockhash: recentBlockhash,
81
		Instructions:    instructions,
82
	}
83
	seen := make(map[PublicKey]struct{})
84
	accounts := make([]*AccountMeta, 0)
85
	accountKeyIndex := make(map[PublicKey]int)
86
	for _, instr := range instructions {
87
		for _, acct := range instr.Accounts {
88
			if _, ok := seen[acct.PubKey]; !ok {
89
				seen[acct.PubKey] = struct{}{}
90
				accounts = append(accounts, &acct)
91
			}
92
		}
93
		if _, ok := seen[instr.ProgramID]; !ok {
94
			seen[instr.ProgramID] = struct{}{}
95
			accounts = append(accounts, &AccountMeta{
96
				PubKey:     instr.ProgramID,
97
				IsSigner:   false,
98
				IsWritable: false,
99
			})
100
		}
101
	}
102

103
	// Sort the accounts
104
	sort.SliceStable(accounts, func(i, j int) bool {
105
		ai := accounts[i]
106
		aj := accounts[j]
107
		if ai.PubKey == feePayer {
108
			return true
109
		}
110
		if ai.IsSigner != aj.IsSigner {
111
			return ai.IsSigner
112
		}
113
		if ai.IsWritable != aj.IsWritable {
114
			return ai.IsWritable
115
		}
116

117
		return false
118
	})
119
	message.AccountKeys = make([]PublicKey, len(accounts))
120
	for i, acc := range accounts {
121
		accountKeyIndex[acc.PubKey] = i
122
	}
123
	for i, inst := range message.Instructions {
124
		// Assign program ID index
125
		message.Instructions[i].ProgramIDIndex = uint8(accountKeyIndex[inst.ProgramID])
126
	}
127

128
	// Initialize message header
129
	for i, acc := range accounts {
130
		if acc.IsSigner {
131
			message.Header.NumRequiredSignatures++
132
			if !acc.IsWritable {
133
				message.Header.NumReadonlySignedAccounts++
134
			}
135
		} else {
136
			if !acc.IsWritable {
137
				message.Header.NumReadonlyUnsignedAccounts++
138
			}
139
		}
140

141
		// Populate account address array
142
		message.AccountKeys[i] = acc.PubKey
143
	}
144

145
	return &Transaction{
146
		Message: message,
147
	}, nil
148
}
149

150
// SerializeMessage serializes a message to byte array
151
func SerializeMessage(msg Message) ([]byte, error) {
152
	buf := new(bytes.Buffer)
153

154
	// Serialize message header
155
	buf.WriteByte(msg.Header.NumRequiredSignatures)
156
	buf.WriteByte(msg.Header.NumReadonlySignedAccounts)
157
	buf.WriteByte(msg.Header.NumReadonlyUnsignedAccounts)
158

159
	// Serialize account keys with compact-array format
160
	// Length as compact-u16
161
	err := EncodeCompactU16Length(buf, len(msg.AccountKeys))
162
	if err != nil {
163
		return nil, fmt.Errorf("failed to encode account keys length: %w", err)
164
	}
165
	for _, acct := range msg.AccountKeys {
166
		buf.Write(acct[:])
167
	}
168

169
	// Serialize recent blockhash
170
	buf.Write(msg.RecentBlockhash[:])
171

172
	// Serialize instructions
173
    // Each instruction is represented in the CompiledInstruction format
174
	// First, number of instructions as compact-u16
175
	err = EncodeCompactU16Length(buf, len(msg.Instructions))
176
	if err != nil {
177
		return nil, fmt.Errorf("failed to encode instructions length: %w", err)
178
	}
179
	for _, instr := range msg.Instructions {
180
		// Program ID index
181
		buf.WriteByte(byte(instr.ProgramIDIndex))
182

183
		accountIndexes := []uint8{}
184
		for _, acct := range instr.Accounts {
185
      // Find the index of the account in the account keys array
186
			for i, key := range msg.AccountKeys {
187
				if key == acct.PubKey {
188
					accountIndexes = append(accountIndexes, uint8(i))
189
					break
190
				}
191
			}
192
		}
193
		// Serialize account indexes with compact-array format
194
		err := EncodeCompactU16Length(buf, len(accountIndexes))
195
		if err != nil {
196
			return nil, fmt.Errorf("failed to encode account indexes length: %w", err)
197
		}
198
		for _, idx := range accountIndexes {
199
			buf.WriteByte(idx)
200
		}
201

202
		// Serialize instruction data
203
		err = EncodeCompactU16Length(buf, len(instr.Data))
204
		if err != nil {
205
			return nil, fmt.Errorf("failed to encode instruction data length: %w", err)
206
		}
207
		buf.Write(instr.Data)
208
	}
209

210
	return buf.Bytes(), nil
211
}
212

213
// SerializeTransaction serializes a transaction to byte array
214
func SerializeTransaction(tx *Transaction, senderPrivateKey ed25519.PrivateKey) ([]byte, error) {
215
	serializedMessage, err := SerializeMessage(tx.Message)
216
	if err != nil {
217
		return nil, fmt.Errorf("failed to serialize message: %w", err)
218
	}
219
	signature, err := SignTransaction(serializedMessage, senderPrivateKey)
220
	if err != nil {
221
		return nil, fmt.Errorf("failed to sign transaction: %w", err)
222
	}
223
	tx.Signatures = append(tx.Signatures, signature)
224
	signatureCount := &bytes.Buffer{}
225
	// Serialize signature array
226
	err = EncodeCompactU16Length(signatureCount, len(tx.Signatures))
227
	if err != nil {
228
		return nil, fmt.Errorf("failed to encode signature count: %w", err)
229
	}
230
	serializedTransaction := make([]byte, 0, signatureCount.Len()+signatureCount.Len()*64+len(serializedMessage))
231
	serializedTransaction = append(serializedTransaction, signatureCount.Bytes()...)
232
	for _, sig := range tx.Signatures {
233
		serializedTransaction = append(serializedTransaction, sig[:]...)
234
	}
235
	serializedTransaction = append(serializedTransaction, serializedMessage...)
236

237
	return serializedTransaction, nil
238
}
239

240
// EncodeCompactU16Length encodes a length as a compact u16
241
// https://github.com/solana-labs/solana/blob/2ef2b6daa05a7cff057e9d3ef95134cee3e4045d/web3.js/src/util/shortvec-encoding.ts
242
func EncodeCompactU16Length(buf *bytes.Buffer, len int) error {
243
	if len < 0 || len > math.MaxUint16 {
244
		return fmt.Errorf("length %d out of range", len)
245
	}
246
	rem_len := len
247
	for {
248
		elem := uint8(rem_len & 0x7f)
249
		rem_len >>= 7
250
		if rem_len == 0 {
251
			buf.WriteByte(elem)
252
			break
253
		} else {
254
			elem |= 0x80
255
			buf.WriteByte(elem)
256
		}
257
	}
258
	return nil
259
}
260

261
// SignTransaction signs a transaction with a private key using ed25519
262
func SignTransaction(serializedTransaction []byte, privateKey ed25519.PrivateKey) (Signature, error) {
263
	signature, err := privateKey.Sign(rand.Reader, serializedTransaction, crypto.Hash(0))
264
	if err != nil {
265
		return Signature{}, fmt.Errorf("failed to sign transaction: %w", err)
266
	}
267
	var sig Signature
268
	copy(sig[:], signature)
269
	return sig, nil
270
}
271

272
type RPCResponse struct {
273
	JsonRPC string `json:"jsonrpc"`
274
	ID      int    `json:"id"`
275
	Result  struct {
276
		Context struct {
277
			Slot uint64 `json:"slot"`
278
		} `json:"context"`
279
		Value struct {
280
			Blockhash string `json:"blockhash"`
281
		} `json:"value"`
282
	} `json:"result"`
283
	Error *struct {
284
		Code    int    `json:"code"`
285
		Message string `json:"message"`
286
	} `json:"error,omitempty"`
287
}
288

289
// GetLatestBlockhash retrieves the latest blockhash from Solana
290
func GetLatestBlockhash(rpcEndpoint string) ([32]byte, error) {
291
	requestBody := map[string]interface{}{
292
		"id":      1,
293
		"jsonrpc": "2.0",
294
		"method":  "getLatestBlockhash",
295
		"params": []map[string]string{{
296
			"commitment": "processed",
297
		}},
298
	}
299
	jsonData, err := json.Marshal(requestBody)
300
	if err != nil {
301
		return [32]byte{}, fmt.Errorf("failed to marshal request: %v", err)
302
	}
303
	req, err := http.NewRequest("POST", rpcEndpoint, bytes.NewBuffer(jsonData))
304
	if err != nil {
305
		return [32]byte{}, fmt.Errorf("failed to create request: %v", err)
306
	}
307
	req.Header.Set("Content-Type", "application/json")
308
	client := &http.Client{}
309
	resp, err := client.Do(req)
310
	if err != nil {
311
		return [32]byte{}, fmt.Errorf("failed to send request: %v", err)
312
	}
313
	defer resp.Body.Close()
314
	body, err := io.ReadAll(resp.Body)
315
	if err != nil {
316
		return [32]byte{}, fmt.Errorf("failed to read response: %v", err)
317
	}
318
	var rpcResponse RPCResponse
319
	if err := json.Unmarshal(body, &rpcResponse); err != nil {
320
		return [32]byte{}, fmt.Errorf("failed to parse response: %v", err)
321
	}
322
	if rpcResponse.Error != nil {
323
		return [32]byte{}, fmt.Errorf("RPC error: %d - %s",
324
			rpcResponse.Error.Code, rpcResponse.Error.Message)
325
	}
326
	decoded, err := base58.Decode(rpcResponse.Result.Value.Blockhash)
327
	if err != nil {
328
		return [32]byte{}, fmt.Errorf("failed to decode blockhash: %v", err)
329
	}
330
	if len(decoded) != 32 {
331
		return [32]byte{}, fmt.Errorf("invalid blockhash length: %d", len(decoded))
332
	}
333
	var result [32]byte
334
	copy(result[:], decoded)
335
	return result, nil
336
}

A lot things going on here, I'll explain some key functions:

• Arrow
  PublicKeyFromBase58(): Since the string representation of a public key is in base58 format, we need a function to convert it to a 32-byte array.
• Arrow
  CreateTransferInstruction(): Create a transfer instruction. It will contain the sender and receiver accounts and the amount of SOL to transfer.
• Arrow
  NewTransaction(): Create a new transaction. It will contain the Message part of the transaction. Account addresses are aggregated and sorted by their privileges. The message header is then populated based on the accounts.
• Arrow
  SerializeMessage(): Serialize a Message to a byte array. It will serialize the message header, account keys, recent blockhash, and instructions sequentially.
• Arrow
  SerializeTransaction(): Serialize a transaction to a byte array. It will serialize Message, sign the Message with the sender's private key, and append the serialized signature array and message to the final byte array.

Finally, let's create the main function and use the functions we just created:

1
// main.go
2
package main
3

4
import (
5
	"encoding/base64"
6
	"fmt"
7
	"log"
8

9
	"github.com/mr-tron/base58"
10
	"golang.org/x/crypto/ed25519"
11
)
12

13
func main() {
14
	senderPubKey := MustPublicKeyFromBase58("ENTER_SENDER_PUBKEY_HERE")
15
	receiverPubKey := MustPublicKeyFromBase58("ENTER_RECEIVER_PUBKEY_HERE")
16
	senderPrivateKeyBytes, err := base58.Decode("ENTER_SENDER_PRIVATE_KEY_HERE")
17
	if err != nil {
18
		log.Fatalf("Invalid private key provided: %v", err)
19
	}
20
	if len(senderPrivateKeyBytes) != ed25519.PrivateKeySize {
21
		log.Fatalf("Private key must be %d bytes", ed25519.PrivateKeySize)
22
	}
23
	senderPrivateKey := ed25519.PrivateKey(senderPrivateKeyBytes)
24

25
	amountSOL := 0.001
26
	lamports := uint64(amountSOL * 1e9) // 1 SOL = 1e9 lamports
27

28
	transferInstr := CreateTransferInstruction(senderPubKey, receiverPubKey, lamports)
29
	recentBlockhash, err := GetLatestBlockhash("https://api.mainnet-beta.solana.com")
30
	if err != nil {
31
		log.Fatalf("Failed to get recent blockhash: %v", err)
32
	}
33
	tx, err := NewTransaction([]Instruction{transferInstr}, recentBlockhash, senderPubKey)
34
	if err != nil {
35
		log.Fatalf("Failed to create transaction: %v", err)
36
	}
37
	serializedTx, err := SerializeTransaction(tx, senderPrivateKey)
38

39
	fmt.Printf("Base64-encoded transaction: %s\n", base64.StdEncoding.EncodeToString(serializedTx))
40
}

Nothing fancy here, it just defines necessary variables, creates a transfer instruction, gets the latest blockhash, creates a transaction, serializes it, and prints the base64-encoded transaction. Remember to replace the placeholders with your own values if you want to run the code.

That's it! We've just created a simple application for transfering SOL between. You can run the code with the following command:

1
go run .

Library alternative

Don't want to reinvent the wheel or believe that the code above can work properly? Here's the code that uses Solana Go SDK to achieve the same result:

1
package main
2

3
import (
4
	"context"
5
	"encoding/base64"
6
	"fmt"
7

8
	"github.com/gagliardetto/solana-go"
9
	"github.com/gagliardetto/solana-go/programs/system"
10
	"github.com/gagliardetto/solana-go/rpc"
11
)
12

13
func main() {
14
	receiverPubKey := solana.MustPublicKeyFromBase58("ENTER_RECEIVER_PUBKEY_HERE")
15
	senderPrivateKey := solana.MustPrivateKeyFromBase58("ENTER_SENDER_PRIVATE_KEY_HERE")
16
	signers := []solana.PrivateKey{senderPrivateKey}
17
    client := rpc.New("https://api.mainnet-beta.solana.com")
18
	res, err := client.GetLatestBlockhash(context.Background(), rpc.CommitmentProcessed)
19
    if err != nil {
20
      panic(err)
21
    }
22
    recentBlockhash := res.Value.Blockhash
23

24
	amountSOL := 0.001
25
	lamports := uint64(amountSOL * 1e9) // 1 SOL = 1e9 lamports
26
	tx, err := solana.NewTransaction(
27
		[]solana.Instruction{
28
			system.NewTransferInstruction(
29
				lamports,
30
				senderPrivateKey.PublicKey(),
31
				receiverPubKey,
32
			).Build(),
33
		},
34
		recentBlockhash,
35
		solana.TransactionPayer(senderPrivateKey.PublicKey()),
36
	)
37
	if err != nil {
38
		panic(err)
39
	}
40
	_, err = tx.Sign(
41
		func(key solana.PublicKey) *solana.PrivateKey {
42
			for _, payer := range signers {
43
				if payer.PublicKey().Equals(key) {
44
					return &payer
45
				}
46
			}
47
			return nil
48
		},
49
	)
50
	serializedTx, err := tx.MarshalBinary()
51
	if err != nil {
52
		panic(err)
53
	}
54

55
	fmt.Printf("Base64-Encoded Tx from Solana-go: %s\n", base64.StdEncoding.EncodeToString(serializedTx))
56
}
57

If you want the exact same result between the two code snippets, you need to hard-code the recent blockhash, as it changes every time you run the code.

Conclusion

I hope this article has helped you understand the structure of a Solana transaction and how to create one without any SDKs or libraries. IMO, this knowledge is essential for building more complex applications on Solana. In the next article, we'll create an application that swaps tokens on the most popular DEX on Solana, Raydium. Stay tuned!