TaffyDB - The JavaScript Database

TaffyDB is an opensouce library that brings database features into your JavaScript applications.

Introduction

How you ever noticed how JavaScript object literals look a lot like records? And that if you wrap a group of them up in an array you have something that looks a lot like a database table? TaffyDB is a libary to bring powerful database funtionality to that concept and rapidly improve the way you work with data inside of JavaScript.

What makes it sticky

• Extremely fast
• Powerful JavaScript centric data selection engine
• Database inspired features such as count, update, and insert
• Robust cross browser support
• Easily extended with your own functions
• Compatible with any DOM library (jQuery, YUI, Dojo, etc)

Creating a JavaScript Database

Let's create a database containing a collection of posts one might find on service like Twitter or Facebook.

var newsfeed = TAFFY([
	{"id":1,"user":"kmart77","stars":3,"text":"JavaScript is a meal best served cold"},
	{"id":2,"user":"lukeisyourfather","stars":5,"text":"Anyone want to meet tonight at Fado?"},
	{"id":3,"user":"bemine","stars":3,"text":"I reallylike the new prirotity inbox #gmail"},
	{"id":4,"user":"keepit","stars":3,"text":"Laughing at this weeks fails video."},
	{"id":5,"user":"piegirl22","stars":3,"text":"It is impossible to change your clothes in a public bathroom. Ick."},
	{"id":6,"user":"kmart77","stars":3,"text":"@kmart77: Have fun in New York!"},
	{"id":7,"user":"justinb","stars":5,"text":"Please set the noise level to earthquake"}
]);

Filtering for data

Now we have a TaffyDB collection containing the JSON objects (records) we passed in. Using the extensive API we can start accessing and working with our data.

newsfeed().count() // returns 7
newsfeed.sort("user") // sort records by username

But the real power comes into play once we start filtering for records. Think of this filtering as using data selectors (if you know jQuery) or building the a database query "where" clause (if you know databases).

newsfeed({user:"kmart77"}).count() // returns 1
// filter for posts where user is equal to kmart77

Keeping this JavaScript centric, we use an object literal to do the record lookup. No ugly string concatenations here.

Now that we understand basic filters lets look at some of the options we have for filtering

newsfeed({text:{ends:"#gmail"}}).count() // returns 1
// filter for posts where text ends with #gmail
// you can also use starts

newsfeed({stars:{gt:3}}).count() // returns 2
// filter for posts where the star rating is greater than 3

There are a lot more types of filtering including options to ignore case, use functions or regular expressions, and traverse objects within your database.

Working with data

When you filter for data you create a query. If you assign this query to a variable you can pass it around. It is not static and the data will be updated if other logic changes the underling records.

var kmart77 = newsfeed({user:"kmart77"});
// kmart77 is now a first class variable and can be passed to functions

So now that you have your query lets look at some of the methods you can use to work with the data it has access to.

kmart77.each(function (r) {
    alert(r.text);
}) // for each record, call the function you've provided and alert the text of the post
kmart77.select(text) // returns an array with the text of each post

var kmart77_5Star = kmart77.filter({stars:5}); // apply another filter

kmart77_5Star.count() // returns 0
kmart77.count() // still returns 1

Modifying data

Let's make a basic change to a record:

newsfeed({id:"1"}).update({stars:5});
// updates the star rating, returns a query object

Now let's insert another post:

newsfeed.insert({"id":8,"user":"justinb","stars":3,"text":"Have a great weekend everyone!"});
//inserts a new record

Finally let's do a remove:

newsfeed({user:"kmart77"}).remove();
//removes all records for user kmart77

Downloads

Get it on github.

CDNs

Beginner's Guide

TaffyDB is very easy to get started with. This brief turtoial will introduce you to a few of the core concepts and should be enough to get started even if you aren't already a web developer.

Step 1: Creating a quick sandbox

It is recmmended that you download and use your own copy of TaffyDB, but for the purposes of playing around you can use the latest version right off of GitHub. If you have never created a web page before you can simply create a text document, include the code below, and save it to your desktop as test.html.

<html>
	<head>
		<script src="https://github.com/typicaljoe/taffydb/raw/master/taffy.js"></script>
	</head>
	<body>
	</body>
				
</html>

The script tag imports TaffyDB right onto the page.

Step 2: Your First DB

Let's create a DB to store some city information. You'll need another script tag and you'll be writing JavaScript.

<html>
	<head>
		<script src="https://github.com/typicaljoe/taffydb/raw/master/taffy.js"></script>
		<script>

		var cities = TAFFY();

		</script>
	</head>
	<body>
	</body>
				
</html>

Step 3: Adding records

You can add records on creation and via the .insert() method. Below we are doing both.

<html>
	<head>
		<script src="https://github.com/typicaljoe/taffydb/raw/master/taffy.js"></script>
		<script>

		var cities = TAFFY([{name:"New York",state:"WA"},{name:"Las Vegas",state:"NV"},{name:"Boston",state:"MA"}]);
	
		cities.insert({name:"Portland",state:"OR"});

		</script>
	</head>
	<body>
	</body>
				
</html>

Step 4: Your first query

Our city db is itself a function that accepts a filter object (an object that will be matched to records). Let's query for Boston. We will alert the .count() method to see our results.

<html>
	<head>
		<script src="https://github.com/typicaljoe/taffydb/raw/master/taffy.js"></script>
		<script>

		var cities = TAFFY([{name:"New York",state:"WA"},{name:"Las Vegas",state:"NV"},{name:"Boston",state:"MA"}]);
	
		cities.insert({name:"Portland",state:"OR"});

		alert(cities({name:"Boston"}).count());

		</script>
	</head>
	<body>
	</body>
				
</html>

Step 5: Chaining methods

You can chain together. Below we will get the first two records from the DB and use the .each() method to alert their names.

<html>
	<head>
		<script src="https://github.com/typicaljoe/taffydb/raw/master/taffy.js"></script>
		<script>

		var cities = TAFFY([{name:"New York",state:"WA"},{name:"Las Vegas",state:"NV"},{name:"Boston",state:"MA"}]);
	
		cities.insert({name:"Portland",state:"OR"});

		cities().limit(2).each(function (r) {alert(r.name)});

		</script>
	</head>
	<body>
	</body>
				
</html>

Step 6: Updating data

You may have noticed that New York is in the wrong state. Let's fix that and alert the new state.

<html>
	<head>
		<script src="https://github.com/typicaljoe/taffydb/raw/master/taffy.js"></script>
		<script>

		var cities = TAFFY([{name:"New York",state:"WA"},{name:"Las Vegas",state:"NV"},{name:"Boston",state:"MA"}]);
	
		cities.insert({name:"Portland",state:"OR"});

		cities({name:"New York"}).update({state:"NY"});

		alert(cities({name:"New York"}).first().state);


		</script>
	</head>
	<body>
	</body>
				
</html>

Step 7: Sorting

Sorting is a big part of TaffyDB using the .order() method.

<html>
	<head>
		<script src="https://github.com/typicaljoe/taffydb/raw/master/taffy.js"></script>
		<script>

		var cities = TAFFY([{name:"New York",state:"WA"},{name:"Las Vegas",state:"NV"},{name:"Boston",state:"MA"}]);
	
		cities.insert({name:"Portland",state:"OR"});

		alert(cities().order("name").first().name);

		</script>
	</head>
	<body>
	</body>
				
</html>

What's next

There are a lot of other methods you can use against your data. Some highlights include .remove() to delete records, .get() to get an array of records, and .supplant to merge records with a string template. Continue to browse the docs for more ways to use TaffyDB.

Mailing List

Get your questions answered. Tell us what you have built. Help others leverage TaffyDB.

Extending TaffyDB

TaffyDB is easy to extend. Simply use the extend method to add a new method to all TaffyDB collections on your page.

Here is an example that creates an "avg" method that takes a column and returns the avg value.

// Create a new empty database
TAFFY.extend("avg",function (c) {
	// This runs the query or returns the results if it has already run
	this.context({
           results: this.getDBI().query(this.context())
    });
    // setup the sum
    var total = 0;
    // loop over every record in the results and sum up the column.
    TAFFY.each(this.context().results,function (r) {
		total = total + r[c];
    })
    
    // divide the total by the number of records and return
    return total/this.context().results.length;
});

What can you build?

Running queries against your database

The heart of TaffyDB and any database is running queries against your data. This is done after creation of your database by calling the root function and building Filter Objects.

// Create a new empty database
var db = TAFFY();

// Run a query against the DB to return all rows
db();

// Real world example - remove all records
db().remove();

Looking up individual records

Every object within a TaffyDB collection has a ___id value set by TaffyDB. This value is not intended for you to know, but is useful when building dynamic applications. This can be used to lookup a record by passing it into the root function as a string.

// Looks up a record based on its id
db("T000008R000002");

// Real world example - update records "status" column
db("T000008R000002").update({status:"Active"});

This also works if you have a copy of the whole record.

// get the first record
var firstRecord = db().first();

// look up this record again
db(firstRecord);

// Real world example - update records "status" column
db(firstRecord).update({status:"Active"});

Using functions

To give you full control over the results of your query you can always pass in a function. Just have it return true if you want the record in your final results.

// functional example, returns all records
db(function () {
	return true;
});


// Real world example - function returns records with a status of active
db(function () {
	return (this.status == "Active") ? true : false;
});

Basic queries

TaffyDB uses a very JavaScript centric Filter Object for looking up queries. There is no string concatenation and you can quickly compose these by hand or dynamically from within your app. The filter object is compared against each record using some business rules and if it passes the record remains in the results set.

The most common Filter Object is used simply to check if a column is equal to value.

// does a match for column and value
db({column:"value"});


// Real world example - records with a status of active
db({status:"Active"});

This is the short form of this

// does a match for column and value
db({column:{is:"value"}});


// Real world example - records with a status of active
db({status:{is:"Active"}});

The "is" part of this expressions can be swapped out for a variety of other comparisons as listed below.

Using !(bang)

For any comparison operator you can quote it and add a ! sign to reverse the meaning.

// does a match for column that is not a value
db({column:{"!is":"value"}});

// Real world example - records with a status other than of active
db({status:{"!is":"Active"}});

Adding additional filters

Using the almighty comma you can add additional lookups to you Filter Object.

// does a match for column that is a value and column2 is a value
db({column:"value",column2:"value"});

// Real world example - records with a status of active and a role of admin
db({status:"Active",role:"Admin"});

You can also pass in additional Filter Objects into the function

// does a match for column that is a value and column2 is a value
db({column:"value"},{column2:"value"});

// Real world example - records with a status of active and a role of admin
db({status:"Active"},{role:"Admin"});

Using arrays for IN and OR

In a database you can use "in" to pass in a collection of values to compare against. This is possible in TaffyDB via the array.

// does a match for column that is one of two values
db({column:["value","value2"]);

// Real world example - records with a status of active or pending
db({status:["Active","Pending"]});

You can also pass in an array of Filter Objects with each one being treated as a logical OR.

// does a match for column that is one of two values
db([{column:"value"},{column:"value2"}]);

// Real world example - records with a status of active or pending
db([{status:"Active"},{status:"Pending"}]);

Bringing it all together

A real world example of a complex query.

// return records where the role is Admin and the status is Active. 
// Also return records where the role is Admin, the status is Pending, and the manager_review is true

db({role:"Admin"},[{status:"Active"},{status:"Pending",manager_review:true}]);

Comparison Operators

In addition to the default "is" operator there are a lot of other operators you can use to lookup records.

Creating your database

You use the global TAFFY function to create a new database.

// Create a new empty database
var db = TAFFY();

// Create a new database a single object (first record)
var db = TAFFY({record:1,text:"example"})

// Create a new database using an array
var db = TAFFY([{record:1,text:"example"}])

// Create a new database using a JSON string
var db = TAFFY('[{"record":1,"text":"example"}]')

DB Queries

Once you have a database you'll be able to call your variable as function. This will setup a query and return a collection of methods to work with the results of that query.

db(); // returns all rows

DB Methods

In addition to the query function you have a couple of top level methods to use.

db.insert({column:"value"}); // inserts one record
						
db.insert({column:"value"},false); // insert but do not call onInsert event

db.order("columnname"); // orders collection by columnname

db.store("name"); // starts storing records in local storage

db.settings({template:{show:true}}); // sets the template to have a show value set to true by default on all records

db.settings({onUpdate:function () {alert(this)}}); // sets the onUpdate event

					

Query Object Methods

Any call to data collections root function will return a new query object. The methods below can be used to interact with the data.

db().update({column:"value"}); // sets column to "value" for all matching records

db().update("column","value"); // sets column to "value" for all matching records

db().update(function () {this.column = "value";return this;}); // sets column to "value" for all matching records


db().update({column:"value"},false); // update but do not call onUpdate event

db().remove(); // removes all matching records from the database
						
db().remove(true); // removes but do not call onRemove event

db().filter({column:value});
						
db().filter({column:value}).count();

collection({column:{gte:value}}).filter({column:{lte:value}}).count();
					

db().order("col1"); // sorts by col1 using a logical sort

db().order("col1 asec, col2 asec"); // sorts by col1 then col2

db().order("col1 desc"); // sorts by col1 descending

db().order("col1 logicaldesc"); // sorts by col1 using a logical sort descending
					

db().limit(15); // Limits returned results to first 15

db().start(15); // Starts returning results at record 15

db().each(function (record,recordnumber) {
	alert(record["balance"]);
}); // alerts the value of the balance column for each record

db().map(function (record,recordnumber) {
	return record.balance*0.2;
}); // returns an array of numbers that are each 20% of the actual balance for each record

db().callback(function () {
    alert(this.count()); // alert count of matching records					
});

db().get(); // returns an array of all matching records

db().stringify(); // returns a JSON array of all matching records

db().first(); // returns the first record out of all matching records.

db().last(); // returns the last record out of all matching records.

db().last("balance"); // returns the sum of the "balance" column.

db().min("balance"); // returns the lowest value of the "balance" column.

db().max("balance"); 
// returns the highest value of the "balance" column.

db().select("charges","credits","balance"); 
// returns an array of arrays in this format [[-24,20,-4]]

db().select("balance"); 
// returns an array of values in this format [-4,6,7,10]
					

db().distinct("title","gender"); 
// returns an array of arrays in this format [["Mr.","Male"],["Ms.","Female"]]

db().distinct("title"); 
// returns an array of values in this format ["Mr.","Ms.","Mrs."]

db().supplant("<tr><td>{balance}</td></tr>"); 
// returns a string in this format "<tr><td>-4</td></tr><tr><td>6</td></tr>"

db().supplant("<tr><td>{balance}</td></tr>",true); 
// returns an array of strings format ["<tr><td>-4</td></tr>","<tr><td>6</td></tr>"]