Modeling

Core Patterns

This guide documents canonical usage patterns for the Go TableTheory SDK, designed to be copy-pasted into your application.

TypeScript and Python ship their own runtime-specific core-pattern guides as sibling package surfaces in the shared TheoryCloud TableTheory subtree. This page is the Go patterns guide.

Lambda Optimization

Problem: AWS Lambda functions suffer from “cold starts” if connections are re-established on every invocation. Solution: Use NewLambdaOptimized in the global scope or init() function.

// ✅ CORRECT: Global initialization
var db *tabletheory.LambdaDB

func init() {
    var err error
    // Initialize once during cold start
    db, err = tabletheory.NewLambdaOptimized()
    if err != nil {
        panic(err)
    }
}

func handler(ctx context.Context) error {
    // db is ready to use immediately
    return db.Model(&User{}).Create()
}

When a function needs a shorter cleanup window than the default 1 second buffer, configure that buffer at cold start and derive an invocation-scoped DB from the Lambda context inside the handler:

// ✅ CORRECT: configure once, derive per invocation
var db *tabletheory.LambdaDB

func init() {
    base, err := tabletheory.LambdaInit(&User{})
    if err != nil {
        panic(err)
    }

    db = base.WithLambdaTimeoutConfig(tabletheory.LambdaTimeoutConfig{
        Buffer: 500 * time.Millisecond,
    })
}

func handler(ctx context.Context) error {
    // WithLambdaTimeout returns core.DB so *LambdaDB satisfies
    // core.ExtendedDB; the derived handle is typed as core.DB.
    invocationDB := db.WithLambdaTimeout(ctx)
    return invocationDB.Model(&User{}).Create()
}

// ❌ INCORRECT: Handler initialization
func handler(ctx context.Context) error {
    db, _ := tabletheory.New(...) // Re-connects every time! 10x slower.
    return db.Model(&User{}).Create()
}

Pagination

Problem: Retrieving large datasets in a single call can exceed DynamoDB limits (1MB) or timeout. Solution: Use Limit() and loop until results are exhausted.

// ✅ CORRECT: Paginated Query
var allUsers []User
lastEvaluatedKey := ""

for {
    var page []User
    // Configure query
    q := db.Model(&User{}).Limit(50)

    // Apply cursor if continuing
    if lastEvaluatedKey != "" {
        q.Cursor(lastEvaluatedKey)
    }

    // Fetch page
    result, err := q.AllPaginated(&page)
    if err != nil {
        log.Fatal(err)
    }

    allUsers = append(allUsers, page...)

    // Check if more pages exist
    if !result.HasMore {
        break
    }
    lastEvaluatedKey = result.NextCursor
}

Optimistic Locking (Versioning)

Problem: Two users update the same item simultaneously, overwriting each other’s changes. Solution: Use a version field and AtVersion condition.

  1. Model Setup: Add a version field.
type Document struct {
    ID      string `theorydb:"pk"`
    Content string
    Version int64  `theorydb:"version"` // Automatically increments on update
}
  1. Update Logic:
// ✅ CORRECT: Guarded Update
doc := &Document{ID: "doc_1", Content: "New Content", Version: 5}

// Fails if current version in DB is not 5
err := db.Model(doc).
    Where("ID", "=", doc.ID).
    WithCondition("Version", "=", 5). // Or use .AtVersion(5) in Transaction
    Update()

if errors.Is(err, customerrors.ErrConditionFailed) {
    // Handle conflict: Fetch latest and retry
}

Batch Operations

Problem: Reading items one by one is slow and inefficient. Solution: Use BatchGet to fetch up to 100 items in parallel.

// ✅ CORRECT: Batch retrieval
var users []User
// Define keys to fetch
keys := []User{
    {ID: "1"},
    {ID: "2"},
    {ID: "3"},
}
// Pass slice of structs with keys set
err := db.Model(&User{}).BatchGet(keys, &users)

DynamoDB Streams

Problem: Processing stream events requires parsing complex DynamoDB JSON. Solution: Use tabletheory.UnmarshalStreamImage to convert stream images to your models.

// ✅ CORRECT: Stream processing
func handleStream(e events.DynamoDBEvent) {
    for _, record := range e.Records {
        if record.EventName == "INSERT" || record.EventName == "MODIFY" {
            var user User
            // Convert Lambda Event Map -> Go Struct
            err := tabletheory.UnmarshalStreamImage(record.Change.NewImage, &user)
            if err != nil {
                log.Println("Parse error:", err)
                continue
            }
            processUser(user)
        }
    }
}

Atomic Transactions

Problem: Need to update multiple items atomically (e.g., bank transfer). Solution: Use db.TransactWrite for ACID guarantees.

// ✅ CORRECT: Transaction
err := db.TransactWrite(ctx, func(tx core.TransactionBuilder) error {
    // 1. Deduct from Sender
    tx.Update(sender, []string{"Balance"})

    // 2. Add to Receiver
    tx.Update(receiver, []string{"Balance"})

    // 3. Record Audit Log
    tx.Put(auditLog)

    return nil
})
// If ANY operation fails, EVERYTHING is rolled back.

Conditional Writes

Problem: Prevent overwriting data if it already exists (idempotency). Solution: Use Create(), which asserts that the partition key does not already exist. Use CreateOrUpdate() only when overwrite/upsert semantics are intentional. .IfNotExists() remains available as an explicit, compatible guard but is redundant with Create().

// ✅ CORRECT: Insert only if ID doesn't exist
err := db.Model(&User{ID: "123"}).Create()

if errors.Is(err, customerrors.ErrConditionFailed) {
    log.Println("User already exists!")
}

Conditional Writes

Problem: Prevent overwriting data if it already exists (idempotency). Solution: Use Create(), which is conditional by default. Choose CreateOrUpdate() explicitly when an existing item may be overwritten.

// ✅ CORRECT: Insert only if ID doesn't exist
err := db.Model(&User{ID: "123"}).Create()

if errors.Is(err, customerrors.ErrConditionFailed) {
    log.Println("User already exists!")
}

Business Value & Use Cases

TableTheory is designed to provide significant business value by improving developer efficiency, reducing operational costs, and enhancing application performance and reliability.

Developer Efficiency & Team Velocity

  • Reduced Boilerplate: TableTheory eliminates approximately 80% of the boilerplate code typically required for DynamoDB interactions with the raw AWS SDK. This frees developers to focus on business logic.
  • Model-aware validation: Go struct types, theorydb tags, and runtime model validation keep marshaling predictable without claiming a generic or compile-time field-name API that does not exist yet.
  • Intuitive API: The fluent, chainable API makes code more readable and easier to maintain, reducing the learning curve for new team members.

Performance & Reliability

  • Sub-15ms Cold Starts: With Lambda-optimized initialization (NewLambdaOptimized, LambdaInit), TableTheory achieves 91% faster cold starts compared to raw AWS SDK usage, crucial for responsive serverless applications.
  • Optimized Memory Usage: TableTheory uses 57% less memory in Lambda environments, contributing to lower execution costs and fewer memory-related issues.
  • Production-Ready Patterns: Built-in support for transactions, conditional writes, and retry logic helps build robust and fault-tolerant applications.

Cost Optimization

  • Reduced RCUs/WCUs: By promoting efficient querying (avoiding scans, using indexes correctly) and batch operations, TableTheory helps minimize consumed Read Capacity Units (RCUs) and Write Capacity Units (WCUs), directly lowering DynamoDB costs.
  • Lower Lambda Costs: Faster cold starts and reduced memory footprint mean Lambda functions run for shorter durations and require less memory, leading to lower compute costs.
  • Faster Development = Lower Project Costs: Increased team velocity translates to projects delivered faster and with fewer resources.

Primary Use Cases

TableTheory is ideal for:

  • Serverless Backends: Building highly scalable and performant APIs with AWS Lambda and API Gateway.
  • Event-Driven Architectures: Processing DynamoDB Streams with struct-based model transformations.
  • High-Throughput Microservices: Services requiring fast, efficient interactions with DynamoDB.
  • Financial & Critical Systems: Leveraging atomic transactions for data consistency.
  • Real-time Data Processing: Applications needing low-latency access to DynamoDB data.

Performance & Cost Optimization

Problem: Unoptimized DynamoDB interactions or Lambda configurations can lead to high costs and slow performance. Solution: Implement strategies for Lambda memory sizing, efficient connection pooling, and careful RCU/WCU management.

Lambda Memory Sizing

Principle: Higher Lambda memory often means more CPU, faster execution, and lower overall cost for compute-bound tasks, despite a higher per-GB-second price.

  • Recommendation: Start with 512MB-1GB for most TableTheory-based Lambda functions. Monitor CPU time and memory usage (using LambdaDB.GetMemoryStats()) to fine-tune.
  • Impact:
    • Memory <= 256MB: May incur higher cold starts due to limited CPU.
    • Memory >= 1024MB: Generally provides optimal performance for typical workloads.

Example (Monitoring Memory Usage):

// ✅ CORRECT: Logging memory stats for optimization
func handler(ctx context.Context) error {
    // ... your business logic ...

    stats := db.GetMemoryStats()
    log.Printf("Lambda Memory Stats: Alloc: %.2fMB, Sys: %.2fMB, Used: %.2f%%",
        stats.AllocatedMB, stats.SystemMB, stats.MemoryPercent)

    // Use these stats to adjust Lambda memory configuration in your CDK/CloudFormation
    return nil
}

Connection Pooling & Reuse

Principle: Reusing HTTP connections and DynamoDB clients across Lambda invocations drastically reduces cold start latency.

  • Recommendation: Always use tabletheory.NewLambdaOptimized() or tabletheory.LambdaInit() in your init() function or global scope.
  • Details: TableTheory’s LambdaDB manages an optimized http.Client with appropriate MaxIdleConns and IdleConnTimeout settings for Lambda’s execution model.
  • Timeout buffer: Use LambdaDB.WithLambdaTimeoutConfig(tabletheory.LambdaTimeoutConfig{Buffer: ...}) during cold start to customize the buffer left before the Lambda hard deadline. Continue to call db.WithLambdaTimeout(ctx) per invocation. WithLambdaTimeout returns core.DB (so *LambdaDB satisfies core.ExtendedDB), while WithLambdaTimeoutConfig stays on the concrete *LambdaDB.
  • No background work: Every TableTheory init and operation path is synchronous or fully joined. Nothing is left running after an init or a call returns, because Lambda freezes the execution environment when the handler returns and a frozen goroutine may resume against an invocation that is already over. Parallel fan-out is allowed only when every worker has finished before the call returns, on success, on the first error, and on context cancellation: Query.ScanAllSegments cancels the remaining segments as soon as one fails and then waits for all of them, and the parallel batch paths wait for their workers on every path. Earlier releases pre-warmed the connection from a detached goroutine that called ListTables; that pre-warm is gone, so a dynamodb:ListTables grant is no longer needed for it.

Example:

// ✅ CORRECT: Global init ensures connection pooling
var db *tabletheory.LambdaDB

func init() {
    db, _ = tabletheory.NewLambdaOptimized()
    db.OptimizeForMemory() // Auto-adjusts internal buffers
}

Read/Write Capacity Unit (RCU/WCU) Management

Principle: Minimize consumed capacity by avoiding full table scans and using efficient access patterns.

  • Recommendation:
    • Avoid Scans: Unless absolutely necessary for infrequent analytics on small tables, never use Scan() for primary access patterns. Always prefer Query() with appropriate Partition and Sort Key conditions.
    • Batch Operations: Use BatchGet, BatchCreate, BatchDelete for multiple items to reduce network overhead and potentially consumed capacity compared to individual operations.
    • Consistent Reads: Only enable ConsistentRead() when strong consistency is strictly required on the base table or an applicable LSI, as it consumes 2x RCUs. Do not use it on GSI access patterns: TableTheory now returns ErrInvalidOperator when an explicit or optimizer-selected GSI is combined with ConsistentRead().
    • GSI Projection: Use KEYS_ONLY or INCLUDE projections on GSIs to reduce the size of items read from the index, minimizing RCU consumption.

Example (Efficient Query vs. Scan):

// ❌ INCORRECT: Expensive full table scan
// Will consume RCU proportional to table size, not result size
db.Model(&Product{}).Where("Category", "=", "electronics").All(&products)

// ✅ CORRECT: Efficient GSI query
// Assumes a GSI 'category-index' with Category as its PK
db.Model(&Product{}).Index("category-index").Where("Category", "=", "electronics").All(&products)

Problem: Prevent overwriting data if it already exists (idempotency). Solution: Use conditional-by-default Create(). Use CreateOrUpdate() only for an intentional upsert.

// ✅ CORRECT: Insert only if ID doesn't exist
err := db.Model(&User{ID: "123"}).Create()

if errors.Is(err, customerrors.ErrConditionFailed) {
    log.Println("User already exists!")
}