Introduction
A public Aleo function leaves enough on-chain evidence for a useful activity feed. You do not need an explorer database or wallet secrets to answer a narrow question such as: "Which transactions called publish on my program?"
We will build that path end to end. A tiny Leo 4.4 program writes public counters, while a TypeScript process asks the Demox Aleo JSON-RPC service for transactions involving one program and function. The process walks every page, resolves each transaction's block height, removes duplicate IDs, and commits one JSON checkpoint only after the whole scan succeeds.
I like the narrow scope. Public activity belongs in a public index. Private records do not. Trying to make one crawler do both usually ends with a view key sitting in a service that never needed it.
What we're building
The Leo program exposes publish(topic, amount). Its mapping accumulates an amount per topic, and its singleton storage counts accepted calls. Both values are public by design.
The indexer keeps data/checkpoint.json with four useful pieces of state:
- The highest chain tip covered by a completed scan.
- Transaction IDs already processed.
- A compact activity feed.
- The program and function used to create that file.
aleoTransactionsForProgram accepts programId, functionName, page, and maxTransactions. The documented response is an array of confirmed-transaction wrappers. Each wrapper contains status, type, index, and a nested transaction; the transaction ID is at transaction.id. The service allows at most 1,000 results per request and also says maxTransactions must be below 1,000, so we will use 250.
Those rows do not carry the block height in the documented schema. We therefore call transaction for each new ID. Its result includes block_id, transaction_id, status, timestamp, and height. A separate latestHeight call gives us a fixed scan horizon.
One awkward fact shapes the design: the docs do not promise a sort order for aleoTransactionsForProgram. Stopping when a page appears older than our checkpoint would be a guess. We scan until the API returns a short page, then use heights for filtering. That costs more calls, but silent gaps are worse than a slower cron job.
Prerequisites
Install Leo 4.4.0, Node.js 20 or newer, and npm. You also need a funded Aleo testnet account if you want to deploy rather than stop after local execution.
The current Demox Testnet Beta JSON-RPC base URL is https://testnetbeta.aleorpc.com. Its public documentation gives no numeric quota or SLA. It says access is free without authentication and may be rate-limited, so the client runs detail requests serially and retries HTTP 429 and transient 5xx responses. If a 429 response includes Retry-After, we obey it.
Create the two projects:
leo new checkpoint_feed
cd checkpoint_feed
mkdir -p indexer/src indexer/data
If checkpoint_feed.aleo is already registered on your target network, choose another program ID and replace it everywhere in the project.
Step-by-step
1. Write the Leo program
Replace src/main.leo with the complete program below.
program checkpoint_feed.aleo {
mapping totals: field => u64;
storage event_count: u64;
fn publish(public topic: field, public amount: u64) -> Final {
return final {
let current: u64 = totals.get_or_use(topic, 0u64);
totals.set(topic, current + amount);
let count: u64 = event_count.unwrap_or(0u64);
event_count = count + 1u64;
};
}
@noupgrade
constructor() {}
}
totals is a public mapping keyed by a field. event_count is singleton storage, which fits one global number better than a fake one-key mapping.
publish takes public inputs because the feed is meant to describe public activity. Mapping and storage operations live inside final { }, where Leo 4.4 runs public on-chain state changes. A storage read returns an option, so the first call reads unwrap_or(0u64) rather than a value the constructor planted. A @noupgrade constructor must stay empty, and it keeps the example's deployed behavior fixed.
Replace program.json too. Here is the full manifest accepted by Leo 4.4:
{
"program": "checkpoint_feed.aleo",
"version": "0.1.0",
"description": "Public activity source for a checkpointed indexer",
"license": "MIT",
"dependencies": null,
"dev_dependencies": null
}
Build it and run the proof-side function locally:
leo build
leo run publish 7field 5u64
leo run publish 42field 9u64
leo run checks the function but does not execute its final block. To change public state, deploy and broadcast an execution:
export NETWORK=testnet
export ENDPOINT=https://api.explorer.provable.com/v1
export PRIVATEKEY='APrivateKey1...'
leo build
leo deploy --broadcast
leo execute publish 7field 5u64 --broadcast
leo execute publish 42field 9u64 --broadcast
Keep the private key out of shell history in real use. The indexer below does not use it.
2. Set up TypeScript
Create indexer/package.json:
{
"name": "checkpoint-feed-indexer",
"private": true,
"type": "module",
"scripts": {
"index": "tsx src/index.ts",
"typecheck": "tsc --noEmit"
},
"devDependencies": {
"@types/node": "^22.0.0",
"tsx": "^4.19.0",
"typescript": "^5.7.0"
}
}
Create indexer/tsconfig.json:
{
"compilerOptions": {
"target": "ES2022",
"module": "NodeNext",
"moduleResolution": "NodeNext",
"strict": true,
"noUncheckedIndexedAccess": true,
"skipLibCheck": true
},
"include": ["src/**/*.ts"]
}
Install the tools:
cd indexer
npm install
3. Build the checkpointed indexer
Create indexer/src/index.ts:
import { mkdir, readFile, rename, writeFile } from "node:fs/promises";
import { dirname, resolve } from "node:path";
const RPC_URL = process.env.RPC_URL ?? "https://testnetbeta.aleorpc.com";
const PROGRAM_ID = process.env.PROGRAM_ID ?? "checkpoint_feed.aleo";
const FUNCTION_NAME = process.env.FUNCTION_NAME ?? "publish";
const CHECKPOINT_FILE = resolve(process.env.CHECKPOINT_FILE ?? "data/checkpoint.json");
const PAGE_SIZE = Number(process.env.PAGE_SIZE ?? "250");
const START_HEIGHT = Number(process.env.START_HEIGHT ?? "0");
const FEED_LIMIT = Number(process.env.FEED_LIMIT ?? "1000");
if (!Number.isInteger(PAGE_SIZE) || PAGE_SIZE < 1 || PAGE_SIZE >= 1000) {
throw new Error("PAGE_SIZE must be an integer from 1 through 999");
}
interface RpcError {
code: number;
message: string;
data?: unknown;
}
interface RpcEnvelope<T> {
jsonrpc: "2.0";
id: number;
result?: T;
error?: RpcError;
}
interface ProgramTransaction {
status: string;
type: string;
index: number;
transaction: {
id: string;
};
}
interface TransactionDetails {
block_id: number;
transaction_id: string;
type: string;
index: number;
status: string;
timestamp: string;
height: string | number;
}
interface Activity {
transactionId: string;
height: number;
blockId: number;
index: number;
status: string;
type: string;
timestamp: string;
}
interface Checkpoint {
version: 1;
programId: string;
functionName: string;
completedHeight: number;
seenTransactionIds: string[];
activity: Activity[];
}
let requestId = 0;
const sleep = (ms: number) => new Promise<void>((ok) => setTimeout(ok, ms));
function retryDelay(response: Response, attempt: number): number {
const header = response.headers.get("retry-after");
if (header) {
const seconds = Number(header);
if (Number.isFinite(seconds)) return Math.max(0, seconds * 1000);
const date = Date.parse(header);
if (Number.isFinite(date)) return Math.max(0, date - Date.now());
}
return Math.min(30_000, 500 * 2 ** attempt);
}
async function rpc<T>(method: string, params: object = {}): Promise<T> {
for (let attempt = 0; attempt < 6; attempt++) {
const response = await fetch(RPC_URL, {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify({ jsonrpc: "2.0", id: ++requestId, method, params }),
signal: AbortSignal.timeout(30_000),
});
const body = await response.text();
if (response.status === 429 || response.status >= 500) {
if (attempt === 5) throw new Error(`${method}: HTTP ${response.status}: ${body}`);
await sleep(retryDelay(response, attempt));
continue;
}
if (!response.ok) throw new Error(`${method}: HTTP ${response.status}: ${body}`);
const envelope = JSON.parse(body) as RpcEnvelope<T>;
if (envelope.error) {
throw new Error(`${method}: RPC ${envelope.error.code}: ${envelope.error.message}`);
}
if (envelope.result === undefined) throw new Error(`${method}: missing result`);
return envelope.result;
}
throw new Error(`${method}: retry loop exhausted`);
}
function emptyCheckpoint(): Checkpoint {
return {
version: 1,
programId: PROGRAM_ID,
functionName: FUNCTION_NAME,
completedHeight: START_HEIGHT - 1,
seenTransactionIds: [],
activity: [],
};
}
async function loadCheckpoint(): Promise<Checkpoint> {
try {
const state = JSON.parse(await readFile(CHECKPOINT_FILE, "utf8")) as Checkpoint;
if (state.programId !== PROGRAM_ID || state.functionName !== FUNCTION_NAME) {
throw new Error("checkpoint belongs to another program or function");
}
return state;
} catch (error) {
if ((error as NodeJS.ErrnoException).code === "ENOENT") return emptyCheckpoint();
throw error;
}
}
async function saveCheckpoint(state: Checkpoint): Promise<void> {
await mkdir(dirname(CHECKPOINT_FILE), { recursive: true });
const temporary = `${CHECKPOINT_FILE}.${process.pid}.tmp`;
await writeFile(temporary, `${JSON.stringify(state, null, 2)}\n`, "utf8");
await rename(temporary, CHECKPOINT_FILE);
}
function asHeight(value: string | number): number {
const height = Number(value);
if (!Number.isSafeInteger(height) || height < 0) {
throw new Error(`invalid transaction height: ${String(value)}`);
}
return height;
}
async function main(): Promise<void> {
const state = await loadCheckpoint();
const committed = new Set(state.seenTransactionIds);
const encountered = new Set<string>();
const settled = new Set<string>();
const staged: Activity[] = [];
const scanTip = asHeight(await rpc<number | string>("latestHeight"));
for (let page = 0; ; page++) {
const rows = await rpc<ProgramTransaction[]>("aleoTransactionsForProgram", {
programId: PROGRAM_ID,
functionName: FUNCTION_NAME,
page,
maxTransactions: PAGE_SIZE,
});
if (!Array.isArray(rows)) throw new Error("transactionsForProgram returned a non-array");
for (const row of rows) {
const transactionId = row.transaction?.id;
if (!transactionId || encountered.has(transactionId)) continue;
encountered.add(transactionId);
if (committed.has(transactionId)) continue;
const detail = await rpc<TransactionDetails>("transaction", { id: transactionId });
const height = asHeight(detail.height);
if (height > scanTip) continue;
settled.add(transactionId);
if (height < START_HEIGHT) continue;
staged.push({
transactionId,
height,
blockId: detail.block_id,
index: detail.index,
status: detail.status,
type: detail.type,
timestamp: detail.timestamp,
});
}
console.log(`page=${page} rows=${rows.length} staged=${staged.length}`);
if (rows.length < PAGE_SIZE) break;
}
const byId = new Map(state.activity.map((item) => [item.transactionId, item]));
for (const item of staged) byId.set(item.transactionId, item);
const activity = [...byId.values()]
.sort((a, b) => b.height - a.height || b.index - a.index)
.slice(0, FEED_LIMIT);
const next: Checkpoint = {
...state,
completedHeight: scanTip,
seenTransactionIds: [...new Set([...state.seenTransactionIds, ...settled])],
activity,
};
await saveCheckpoint(next);
console.log(`committed height=${scanTip} new=${staged.length} feed=${activity.length}`);
}
main().catch((error) => {
console.error(error);
process.exitCode = 1;
});
The first RPC call captures scanTip. Transactions above that height wait for the next run, which gives the checkpoint a clear meaning: every committed file describes one completed scan against one chain horizon.
encountered removes duplicate IDs returned on overlapping or shifting pages. seenTransactionIds makes that decision survive restarts. Previously unseen transactions below the old checkpoint are still accepted; that choice catches provider indexing lag instead of pretending the RPC result was frozen at the earlier scan.
Writes use a temporary file followed by rename. If the process dies on page 17, the old checkpoint remains valid and the next run repeats the scan. Only a complete pass can move completedHeight.
I deliberately store metadata rather than whole transitions or proofs. A public call may contain large encrypted values that add no value to an activity list. More important, the process has no private key or view key, so it cannot drift into wallet surveillance by accident.
4. Run it
Point the indexer at the deployed program and run one pass:
export RPC_URL=https://testnetbeta.aleorpc.com
export PROGRAM_ID=checkpoint_feed.aleo
export FUNCTION_NAME=publish
export START_HEIGHT=0
export PAGE_SIZE=250
export FEED_LIMIT=1000
npm run typecheck
npm run index
cat data/checkpoint.json
For a program deployed late in the chain's life, set START_HEIGHT to its deployment height. The RPC still has to traverse program pages because their ordering is undocumented, but old rows can be marked seen without entering the feed.
Run the command from cron or a systemd timer. Do not run overlapping copies against the same checkpoint file. A single-process file lock or SQLite transaction is the next sensible addition if your scheduler can overlap jobs.
Testing
Start with an empty indexer directory, execute two calls, and run the indexer twice:
rm -f data/checkpoint.json
cd ..
leo execute publish 7field 5u64 --broadcast
leo execute publish 42field 9u64 --broadcast
cd indexer
npm run index
cp data/checkpoint.json /tmp/checkpoint-first.json
npm run index
cmp /tmp/checkpoint-first.json data/checkpoint.json
The first pass should add the two transaction IDs once. The second should report new=0 if the chain tip and provider result have not changed. A changed completedHeight can make cmp differ even with no new activity, so inspect activity when another block landed between runs.
Now test crash safety. Temporarily set RPC_URL to an invalid host after a checkpoint exists, run the command, and confirm the JSON file did not change:
cp data/checkpoint.json /tmp/checkpoint-good.json
RPC_URL=https://invalid.example npm run index || true
cmp /tmp/checkpoint-good.json data/checkpoint.json
For the pagination path, use PAGE_SIZE=1. That forces several requests and makes duplicate handling easy to inspect:
rm -f data/checkpoint.json
PAGE_SIZE=1 npm run index
node -e 'const s=require("./data/checkpoint.json"); console.log(s.activity.map(x=>x.transactionId))'
Every printed transaction ID should be unique. Also compare event_count and totals[7field] through your preferred Aleo explorer or mapping query. A transaction can exist while its final block failed, so public state remains the authority for application state; the feed is an audit trail of transaction activity.
What's next
A JSON file is enough for one process and a modest feed. Move the same commit boundary into SQLite when readers and writers need concurrency: insert activities with a unique constraint on transaction_id, then update the checkpoint in the same database transaction.
The ABI-driven transaction form tutorial is a good companion for generating calls from deployed program metadata. For typed execution and local-node integration tests, continue with Generate a Typed Aleo Client and Test It Against a Local Devnode.
I would add a provider adapter before adding queues. Demox and a self-hosted snarkOS node may disagree on method names or response extras before your indexing logic changes. Keep the normalized Activity type stable and isolate those differences at the network edge.
For high-volume programs, ask for a cursor or a documented newest-first guarantee rather than inferring one. Until the API makes that contract, full pagination plus durable ID deduplication is the boring correct choice. Boring is good when the alternative is a missing transaction.