Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
multi: Add cache layer (#660)
* multi: Add cache layer

This commit adds a cache layer to politeiad and politeiawww.  The
politeiad git repositories serve as the source of truth for proposal
data and a CockroachDB database acts as a cache for proposal data.

politeiad has read and write access to the cache.  politeiawww has only
read access to the cache.  The flow of data is as follows:

1. politeiawww receives a command from a user
2. politeiawww creates a politeiad request for the command and sends it
3. politeiad writes the command data to the git repository and updates
   the cache
4. politeiad returns a response to politeiawww
5. politeiawww reads the cache if needed
6. politeiawww returns a response to the user

The cache is automatically rebuilt on cache version change.  You can
also force the cache to rebuilt by using the flag --buildcache on
politeiad startup.

The politeiad record and all record metadata has been removed from the
politeiawww inventory.  The only thing left in the inventory is decred
plugin data.  The decred plugin data will be added to the cache and
removed from the inventory in a future PR.

Since this commit touched so much of the code for the `/proposals`
endpoints, I began cleaning up `backend.go` and moving the updated
`/proposals` enpoint code to `proposals.go`.

I had to delete all of the tests that relied on the proposal record in
the inventory.  I re-wrote most of these tests in `proposals_test.go`.

* Add vote data to cache

* build plugin cache and fix stuff

* cleanup

* doco and cleanup

* vim playing tricks on me

* readme update

* doco fixes

* votestatus bug fix

* fix nits

* doco fix
  • Loading branch information
lukebp authored and marcopeereboom committed Feb 8, 2019
1 parent f2a7885 commit 076c4f8
Show file tree
Hide file tree
Showing 46 changed files with 6,688 additions and 4,007 deletions.
81 changes: 76 additions & 5 deletions README.md
Expand Up @@ -96,17 +96,24 @@ You can also use the following default configurations:
rpcuser=user
rpcpass=pass
testnet=true
cachehost=localhost:26257
enablecache=true
cacherootcert="~/.cockroachdb/certs/ca.crt"
cachecertdir="~/.cockroachdb/certs"


**politeiawww.conf**:

rpchost=127.0.0.1
rpcuser=user
rpcpass=pass
rpccert="/Users/<username>/Library/Application Support/Politeiad/https.cert"
rpccert="~/.politeiad/https.cert"
testnet=true
paywallxpub=tpubVobLtToNtTq6TZNw4raWQok35PRPZou53vegZqNubtBTJMMFmuMpWybFCfweJ52N8uZJPZZdHE5SRnBBuuRPfC5jdNstfKjiAs8JtbYG9jx
paywallamount=10000000
cachehost=localhost:26257
cacherootcert="~/.cockroachdb/certs/ca.crt"
cachecertdir="~/.cockroachdb/certs"

**Things to note:**

Expand All @@ -118,7 +125,62 @@ things like new user registration, and those settings are also configured within
`politeiawww.conf`. The current code should work with most SSL-based SMTP servers
(but not TLS) using username and password as authentication.

#### 4. Build the programs:
#### 4. Setup CockroachDB or Postgres:

politeiad stores proposal data in git repositories that are regularly backed up
to github and cryptographically timestamped onto the Decred blockchain. The
politeiad git repositories serve as the source of truth for proposal data. In
order to increase performance, a CockroachDB database acts as a cache for
proposal data.

**The cache is not required if you're running just politeiad. politeiad has
the cache disable by default. If you're running the full politeia stack,
politeiad and politeiawww, running the cache is required.**

politeiad has read and write access to the cache. politeiawww has only read
access to the cache. The flow of data is as follows:

1. politeiawww receives a command from a user
2. politeiawww creates a politeiad request for the command and sends it
3. politeiad writes the command data to the git repository and updates the
cache
4. politeiad returns a response to politeiawww
5. politeiawww reads the cache if needed
6. politeiawww returns a response to the user

We use CockroachDB for the cache in the instructions below. CockroachDB is
built to be compatible with Postgres so you can use Postgres for the cache if
you so choose. Using Postgres for the cache has not been thoroughly tested and
bugs may exist.

Install CockroachDB using the instructions found in the [CockroachDB
Documentation](https://www.cockroachlabs.com/docs/stable/install-cockroachdb-mac.html).

Run the following commands to create the CockroachDB certificates required for
running the cache locally. It's ok if the directory `~/.cockroachdb` does not
exist yet. The script will create it. **The directory that you pass into the
script must be the same cockroachdb directory that you use in the politeiad and
politeiawww config files when specifying `cacherootcert` and `cachecertdir`.**

cd $GOPATH/src/github.com/decred/politeia
./cockroachsetup.sh ~/.cockroachdb

You can now start CockroachDB using the command below. Politeia requires that
CockroachDB be running.

cockroach start \
--certs-dir=${HOME}/.cockroachdb/certs \
--listen-addr=localhost \
--store=${HOME}/.cockroachdb/data

If you want to run database commands manually you can do so by opening a sql
shell.

cockroach sql \
--certs-dir=${HOME}/.cockroachdb/certs \
--host localhost \

#### 5. Build the programs:

Go 1.11 introduced [modules](https://github.com/golang/go/wiki/Modules), a new
dependency management approach, that obviates the need for third party tooling
Expand All @@ -144,11 +206,11 @@ The go tool will process the source code and automatically download
dependencies. If the dependencies are configured correctly, there will be no
modifications to the `go.mod` and `go.sum` files.

#### 5. Start the politeiad server by running on your terminal:
#### 6. Start the politeiad server by running on your terminal:

politeiad

#### 6. Download politeiad's identity to politeiawww:
#### 7. Download politeiad's identity to politeiawww:

politeiawww --fetchidentity

Expand All @@ -165,7 +227,7 @@ The result should look something like this:
2018-08-01 22:49:53.929 [INF] PWWW: Identity saved to: /Users/<username>/Library/Application Support/Politeiawww/identity.json
```

#### 7. Start the politeiawww server by running on your terminal:
#### 8. Start the politeiawww server by running on your terminal:

politeiawww

Expand Down Expand Up @@ -219,6 +281,15 @@ When using politeiawww_refclient, the `-use-paywall` flag is true by default. Wh
* Set the user created in the first refclient execution as admin with politeiawww_dbutil.
* Run refclient again with the `email` and `password` flags set to the user created in the first refclient execution.

#### Rebuilding the Cache

The cache will be built automatically on initial startup of politeiad and when
the cache version has changed, but there may also be times during development
that you want to force the cache to rebuild. You can do this by using the
`--buildcache` flag when starting `politeiad`. This will drop all current
tables from the cache, re-create the tables, then populate the cache with the
data that is in the politeiad git repositories.

## Integrated Projects / External APIs / Official URLs

* https://faucet.decred.org - instance of [testnetfaucet](https://github.com/decred/testnetfaucet)
Expand Down
46 changes: 46 additions & 0 deletions cockroachsetup.sh
@@ -0,0 +1,46 @@
#!/usr/bin/env bash
# This script creates the certificates required to run a CockroachDB instance
# locally as a politeiad cache.

set -ex

readonly COCKROACHDB_DIR=$1

if [ "${COCKROACHDB_DIR}" == "" ]; then
>&2 echo "Error: missing argument CockroachDB directory"
exit
fi

# Create cockroachdb directories.
mkdir -p "${COCKROACHDB_DIR}/certs"
mkdir -p "${COCKROACHDB_DIR}/data"

# Create the Certificate Authority certificate and key pair.
cockroach cert create-ca \
--certs-dir="${COCKROACHDB_DIR}/certs" \
--ca-key="${COCKROACHDB_DIR}/ca.key" \

# Create the node certificate and key. These files, node.crt and node.key,
# will be used to secure communication between nodes. You would generate these
# separately for each node with a unique addresses.
cockroach cert create-node localhost \
$(hostname) \
--certs-dir="${COCKROACHDB_DIR}/certs" \
--ca-key="${COCKROACHDB_DIR}/ca.key"

# Create the client certificate and key for the root user. These files,
# client.root.crt and client.root.key, will be used to secure communication
# between the built-in SQL shell and the cluster.
cockroach cert create-client root \
--certs-dir="${COCKROACHDB_DIR}/certs" \
--ca-key="${COCKROACHDB_DIR}/ca.key"

# Create the client certificate and key for the politeiad user.
cockroach cert create-client politeiad \
--certs-dir="${COCKROACHDB_DIR}/certs" \
--ca-key="${COCKROACHDB_DIR}/ca.key"

# Create the client certificate and key for politeiawww user.
cockroach cert create-client politeiawww \
--certs-dir="${COCKROACHDB_DIR}/certs" \
--ca-key="${COCKROACHDB_DIR}/ca.key"

0 comments on commit 076c4f8

Please sign in to comment.