Friday, June 27, 2014

Google I/O 2012 - Go Concurrency Patterns

Google I/O 2012 - Go Concurrency Patterns
Rob Pike

Concurrency is the key to designing high performance network services. Go's concurrency primitives (goroutines and channels) provide a simple and efficient means of expressing concurrent execution. In this talk we see how tricky concurrency problems can be solved gracefully with simple Go code.



For all I/O 2012 sessions, go to https://developers.google.com/io/

Make a set in Go

The Golang docs say:
A set can be implemented as a map with value type bool. Set the map entry to true to put the value in the set, and then test it by simple indexing.
Here is such a set implementation:
package main

import (
    "log"
    "runtime"
)

func main() {
    const SIZE = 10000000
    set := make(map[int]bool, SIZE)
    for i := 0; i < SIZE; i++ {
        set[i] = true
    }
    memStats := runtime.MemStats{}
    runtime.ReadMemStats(&memStats)
    println("Used", memStats.Alloc, "bytes")

    // test the set
    for i := 0; i < SIZE; i++ {
        _, ok := set[i]
        if !ok {
            log.Panic("The set does not contain the expected value!")
        }
    }
    for i := SIZE; i < SIZE*2; i++ {
        _, ok := set[i]
        if ok {
            log.Panic("The set contains an unexpected value!")
        }
    }
}
The memory usage is:
$ go run sets.go
Used 195924120 bytes
It turns out you can make the values to consuming 0 bytes of memory if they will not of type bool, but struct{}:
package main

import (
    "log"
    "runtime"
)

func main() {
    const SIZE = 10000000
    set := make(map[int]struct{}, SIZE)
    for i := 0; i < SIZE; i++ {
        set[i] = struct{}{}
    }
    memStats := runtime.MemStats{}
    runtime.ReadMemStats(&memStats)
    println("Used", memStats.Alloc, "bytes")

    // test the set
    for i := 0; i < SIZE; i++ {
        _, ok := set[i]
        if !ok {
            log.Panic("The set does not contain the expected value!")
        }
    }
    for i := SIZE; i < SIZE*2; i++ {
        _, ok := set[i]
        if ok {
            log.Panic("The set contains an unexpected value!")
        }
    }
}
And you can see that the memory usage with a map of 10,000,000 integers dropped:
$ go run sets.go
Used 177338200 bytes
The difference is 18,585,920 bytes.

Monday, May 5, 2014

Cache me if you can: memcached, caching patterns and best practices

PyCon US 2014, Guillaume Ardaud

Memcached is a popular, blazing fast in-RAM key/object store mainly used in web applications (although it can be used in virtually any software). You will walk out of this talk with a solid understanding of memcached and what it does under the hood, and become familiar with several patterns and best practices for making the most of it in your own Python applications.

Sunday, April 27, 2014

All Your Ducks In A Row: Data Structures in the Standard Library and Beyond

PyCon US 2014, Brandon Rhodes

Why are Python programmers crazy about lists and dictionaries, when other languages tout bitmaps, linked lists, and B+ trees? Are we missing out? Come learn how data structures are implemented on bare metal, how to select the right data structure, how the list and dictionary cover a wide swath of use cases, and when to dip into the Standard Library or a third-party package for an alternative.

Quick Wins for Better Website Security

PyCon US 2014, Dan Callahan

Learn quick, easy, and lesser-known techniques to improve your website's security, protect against session hijacking, and defend against XSS and data injection attacks.

Tuesday, April 22, 2014

How to build a full-stack python

PyCon US 2014, Kate Heddleston

This is a talk about building full-stack python web applications where you manage every part of the application yourself. I will walk through how to setup a production server with your web application code, a local development environment using vagrant, and how to deploy from your local environment to production. I will also walk through python and Django libraries that will make your life easier.

Postgres Performance for Humans

PyCon US 2014, Craig Kerstiens

Postgres Performance for Humans

To many developers the database is a black box. You expect to be able to put data into your database, have it to stay there, and get it out when you query it... hopefully in a performant manner. When its not performant enough the two options are usually add some indexes or throw some hardware at it. We'll walk through a bit of a clearer guide of how you can understand and manage DB performance.