Running mongodb as a replicaSet in Docker (and adding a new SECONDARY and then upgrading from 3.0 to 3.4)

This is a continuation of the previous article on how to run mongodb in docker in a replica set

We start off by a mongodb cluster of two nodes, running in a docker setup like this:

docker-compose.yml

version: '3'
services:
  db01:
    image: mongo:3.0
    volumes:
    - datadb01:/data/db
    - ./etc/mongod.conf:/etc/mongod.conf
    ports:
    - "30001:30001"
    command: ["mongod", "--config", "/etc/mongod.conf", "--port","30001"]
    container_name: db01

  db02:
    image: mongo:3.0
    volumes:
    - datadb02:/data/db
    - ./etc/mongod.conf:/etc/mongod.conf
    ports:
    - "30002:30002"
    command: ["mongod", "--config", "/etc/mongod.conf", "--port","30002"]
    container_name: db02

volumes:
  datadb01:
  datadb02:

Step Three – Add another host to the replication set

Now adding a third one to the config seems straight forward:

version: '3'
services:
  db01:
    image: mongo:3.0
    volumes:
    - datadb01:/data/db
    - ./etc/mongod.conf:/etc/mongod.conf
    ports:
    - "30001:30001"
    command: ["mongod", "--config", "/etc/mongod.conf", "--port","30001"]
    container_name: db01

  db02:
    image: mongo:3.0
    volumes:
    - datadb02:/data/db
    - ./etc/mongod.conf:/etc/mongod.conf
    ports:
    - "30002:30002"
    command: ["mongod", "--config", "/etc/mongod.conf", "--port","30002"]
    container_name: db02

  db03:
    image: mongo:3.0
    volumes:
    - datadb03:/data/db
    - ./etc/mongod.conf:/etc/mongod.conf
    ports:
    - "30003:30003"
    command: ["mongod", "--config", "/etc/mongod.conf", "--port","30003"]
    container_name: db03

volumes:
  datadb01:
  datadb02:
  datadb03:

To add this third machine to the replSet, reconfiguration in the mongo shell is required:

Seems to be as easy as rs.add("db03:30003") (with MongoDB version >=3.0)

A rs.status() check reveals that the third server is part of the cluster.

It stayed in startup states for a little bit of time (no transactions going on in this test environment)…


                {
                        "_id" : 3,
                        "name" : "db03:30003",
                        "health" : 1,
                        "state" : 5,
                        "stateStr" : "STARTUP2",
                        "uptime" : 12,
                }

… but finally managed to start up completely:

{
        "set" : "rs0",
        "date" : ISODate("2017-06-23T10:30:02.225Z"),
        "myState" : 1,
        "members" : [
                {
                        "_id" : 1,
                        "name" : "db01:30001",
                        "health" : 1,
                        "state" : 1,
                        "stateStr" : "PRIMARY",
...
                        "self" : true
                },
                {
                        "_id" : 2,
                        "name" : "db02:30002",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",
...
                        "pingMs" : 0,
                        "configVersion" : 139230
                },
                {
                        "_id" : 3,
                        "name" : "db03:30003",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",
...
                        "pingMs" : 0,
                        "configVersion" : 139230
                }
        ],
        "ok" : 1
}

We have a running cluster on mongodb version 3.0

So, next stop: Update the cluster from 3.0 to 3.4.

Step four and five – update 3.0 to 3.2 to 3.4

((WAIT, you might want to update your applications configuration now as well, see below))

According to the official mongo documentation, this needs to be done in two steps:

3.0 to 3.2: https://docs.mongodb.com/manual/release-notes/3.2-upgrade/#upgrade-a-replica-set-to-3-2

3.2 to 3.4: https://docs.mongodb.com/manual/release-notes/3.4-upgrade-replica-set/#upgrade-replica-set

In both cases, the steps seem to be the same and quite straight forward:

  1. Upgrade secondary members of the replica set
  2. Step down the replica set primary to secondary, so an upgraded one becomes primary
  3. Upgrade the previous primary so all are on the same version

In this case, using docker, the upgrades of the instances should be as easy as changing the version tag in the docker-compose.yml.
So, one at a time:
As my current primary is db01, I’ll start with db02. The change is just a version number in the file, so I’m not pasting the whole file here:

  db02:
    image: mongo:3.2

A docker-compose up -d brought db02 down, replacing it with an updated mongod 3.2 and repeating and watching rs.status(), I could see the machine disapear and the re-sync.
NICE
Repeat it for db03
NICE again

Next step – step down
Running rs.stepDown on the PRIMARY db01 makes db03 turn PRIMARY and leaves db01 a SECONDARY, ready to update to 3.2 as well…

BUT WAIT!

This made me aware of the fact that I forgot to update my application configuration. While I extended the cluster to a 3-host-system, I did not add db03 to the applications mongo server config and the application server’s /etc/host – which I quickly changed at this point.

Changing the db01’s image to 3.2 now and running docker-compose up -d did update the image/container and restart it – but rs.status() made me also aware that – according to their uptime – the other instances seem to have been restarted as well.

So, there must be a way to update/restart single services of docker-compose, right? Let’s check during the upgrade from 3.2 to 3.4

Now that all 3 containers are running the 3.2 image, the SECONDARYs can be updated as well. The line changed in the docker-compose.yml:

version: '3'
services:
  db01:
    image: mongo:3.4
    ...

Now, instead of running a full docker-compose up -d, it seems the way to go is

docker-compose stop db02
docker-compose create db02
docker-compose start db02

A previous docker-compose up -d db01 had an effect on the other servers uptimes as well, so I verified with db02 that this works.

After connection with the mongo shell to the PRIMARY (db03) and sending it a rs.stepDown(), this one is ready to be upgraded as well.

With the stop, create, start sequence, the last container is upgraded to 3.4 as well and the exercise is finished.

Running mongodb as a replicaSet in Docker (and upgrading it from 3.0 to 3.4)

Goal:

This post is about two things in one go:

Prerequesits (or what I have and use for my case):

  •  a virtual machine running docker
  • an application conneting to and using mongodb
  • a db dump for the application

Step One – Configure and boot a single mongo server with docker

This is influenced by the very good article Creating a MongoDB replica set using Docker.

It gives details about the basics (set up containers, start a replica set). My goal is to go a little bit furhter, though. In addtion to what the article suggests, I’d like to have the data of each container in a data volume. And I’d like to use docker-compose to keep the whole setup in order.

Using the version 3 syntax of docker-compose, I come up with a very basic initial file to start from:

version: '3'
services:
  db01:
    image: mongo:3.0
    volumes:
    - datadb01:/data/db
    ports:
    - "30001:27017"

volumes:
  datadb01:

What it does:

  • Use version 3 of the compose syntax
  • define a first db service, based on a mongo image with the version tag 3.0
  • expose the image’s port 27017 on the host as port 30001
  • mount a named data volume datadb01 into the container at /data/db (the default path of MongoDBs data storage)

This can be run with docker-compose up -d and new we have a single instance mongodb running on port 30001, accessible from the outside.

Step Two – Extend the single mongodb instance to become a multi-host replica set

Adding a second host to the configuration is straight forward and requires some copy & paste so the docker-compose.yml file looks like this

version: '3'
services:
  db01:
    image: mongo:3.0
    volumes:
    - datadb01:/data/db
    ports:
    - "30001:27017"
  db02:
    image: mongo:3.0
    volumes:
    - datadb02:/data/db
    ports:
    - "30002:27017"

volumes:
  datadb01:
  datadb02:

To check if the machine is up, I connect to the second mongod instance from another machine with mongo --port 30002. Of course, this is – as of right now – only a separate single instance of mongod and not a replicaSet, as confirmed by a quick check of the replication status:


> rs.status()
{ "ok" : 0, "errmsg" : "not running with --replSet", "code" : 76 }

At this point, I decided to make this another mongo exercise and start the replicaSet with only two servers, import my data, and only later add the third machine.

So, to get this dual-setup running, we need to tell the machines what replicaSet they are part of. This can be done with a command line option on mongod (--replSet), but I wanted to make it more versatile and put some options into a config file for mongo and start the daemon by telling it where to pull the config from.

So, in a subfolder etc, the simple config file etc/mongod.conf is created:

replication:
   oplogSizeMB: 400
   replSetName: rs0

(the oplog size is a random number here and should be correctly adjusted in production environments)

Now we need to map this file into the containers and tell mongod to read it during startup:

version: '3'
services:
  db01:
    image: mongo:3.0
    volumes:
    - datadb01:/data/db
    - ./etc/mongod.conf:/etc/mongod.conf
    ports:
    - "30001:27017"
    command: ["mongod", "--config", "/etc/mongod.conf"]

  db02:
    image: mongo:3.0
    volumes:
    - datadb02:/data/db
    - ./etc/mongod.conf:/etc/mongod.conf
    ports:
    - "30002:27017"
    command: ["mongod", "--config", "/etc/mongod.conf"]

volumes:
  datadb01:
  datadb02:

What we have here now in addition:

  • copy the file /etc/mongod.conf to /etc/mongod.conf in the container
  • start the container with the additional options, resulting in mongod --conf /etc/mongod.conf

Until now, I was under the impression I could spin up a mongo cluster just like that, but some research and this question on stack overflow made me aware that it won’t work withough a little bit of shell command line.

So, let’s init the replSet

To get the set working, we need to define a config in the mongo shell, for exaple like this:

> rs.initiate({
  "_id": "rs0",
  "version": 1,
  "members" : [
   {"_id": 1, "host": "db01:27017"},
   {"_id": 2, "host": "db02:27017"}
  ]
 })

(Note: as the machines connect internally, the internal ports 27017 need to be used, not the exposed ones)

However, to make this work, the containers need to be known as db01 and db02. They autom automatically got a generated name by docker-compose. So the names have to be added in the docker-compose file to be manually set:

version: '3'
services:
  db01:
    image: mongo:3.0
    volumes:
    - datadb01:/data/db
    - ./etc/mongod.conf:/etc/mongod.conf
    ports:
    - "30001:27017"
    command: ["mongod", "--config", "/etc/mongod.conf"]
    container_name: db01

  db02:
    image: mongo:3.0
    volumes:
    - datadb02:/data/db
    - ./etc/mongod.conf:/etc/mongod.conf
    ports:
    - "30002:27017"
    command: ["mongod", "--config", "/etc/mongod.conf"]
    container_name: db02

volumes:
  datadb01:
  datadb02:

After another docker-compose up -d, the config above can be initialized and results in a happy replication set:

> rs.status()
{
        "set" : "rs0",
        "date" : ISODate("2017-06-21T14:38:13.720Z"),
        "myState" : 2,
        "members" : [
                {
                        "_id" : 1,
                        "name" : "db01:27017",
                        "health" : 1,
                        "state" : 1,
                        "stateStr" : "PRIMARY",
                        "uptime" : 161,
                        "optime" : Timestamp(1498055732, 1),
                        "optimeDate" : ISODate("2017-06-21T14:35:32Z"),
                        "lastHeartbeat" : ISODate("2017-06-21T14:38:12.384Z"),
                        "lastHeartbeatRecv" : ISODate("2017-06-21T14:38:12.384Z"),
                        "pingMs" : 0,
                        "electionTime" : Timestamp(1498055736, 1),
                        "electionDate" : ISODate("2017-06-21T14:35:36Z"),
                        "configVersion" : 1
                },
                {
                        "_id" : 2,
                        "name" : "db02:27017",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",
                        "uptime" : 216,
                        "optime" : Timestamp(1498055732, 1),
                        "optimeDate" : ISODate("2017-06-21T14:35:32Z"),
                        "configVersion" : 1,
                        "self" : true
                }
        ],
        "ok" : 1
}

Now it’s time to import some data into the cluster and try to connect my existing application to the new cluster.
It should be noted here that the application is NOT running as part of the docker setup but is intended to connect to the ports exposed.

quick break, have some coffee while we wait for mongoimport to finish

Importing the data to the cluster with a mongoimport shell script on the primary server is not a but problem, but my PHP and old \MongoClient based application seems to have a problem:

MongoConnectionException
No candidate servers found

MongoConnectionException
MongoClient::__construct(): php_network_getaddresses: getaddrinfo failed: Name or service not known

Looks like that fact that using different IPs and ports “on the outside” (the configuration exposed by docker) is not good enough for the php mongo driver.
To circumvent this, let’s try to match internal and external configurations:

First, match up internal and external mongod ports by changing the internal ones:

version: '3'
services:
  db01:
    image: mongo:3.0
    volumes:
    - datadb01:/data/db
    - ./etc/mongod.conf:/etc/mongod.conf
    ports:
    - "30001:30001"
    command: ["mongod", "--config", "/etc/mongod.conf", "--port","30001"]
    container_name: db01

  db02:
    image: mongo:3.0
    volumes:
    - datadb02:/data/db
    - ./etc/mongod.conf:/etc/mongod.conf
    ports:
    - "30002:30002"
    command: ["mongod", "--config", "/etc/mongod.conf", "--port","30002"]
    container_name: db02

volumes:
  datadb01:
  datadb02:

The command is extended to start mongod internally on port 30001 (30002 for db02) while still exposing it on the same port.

Then the hostnames db01/db02 are added to the application server’s /etc/hosts so there is no problem resolving the name

192.168.10.20    db01
192.168.10.20    db02

After another docker-compose up -d, the changed configurations is applied; however, this breaks our cluster!! The primary and secondary have changed their internal ports, so the cluster connection is lost.

To tell the replSet about this, we need to reconfigure the cluster with the changes:

rs.reconfig({
  "_id": "rs0",
  "version": 1,
  "members" : [
   {"_id": 1, "host": "db01:30001"},
   {"_id": 2, "host": "db02:30002"}
  ]},
  {"force": true }
)

After that, my application is able to connect to the new cluster and everything seems to be fine.

A note on the php config, though:

The MongoClient configuration needs some help to know that there is a cluster and where to perform read/write operations, so the following additional information is necessary:

server-config:
  'mongodb://db01:30001,db02:30002'

client-options:
 readPreference: primary
 replicaSet: rs0

Where and how to put this depends in the individual application, in my case with doctrine_mongodb it looks something like this in the symfony’s config.yml:

doctrine_mongodb:
    connections:
        default:
            server: "mongodb://db01:30001,db02:30002"
            options:
                db: "%mongo_database%"
                readPreference: primary
                replicaSet: rs0

As this article got a little bit longer than expected, the rest will follow in another one

Symfony2 and me – let’s be friends – Part 3

So far, I have a base project with Propel as my ORM, connected to an exisitng DB, re-engineered the schema and could start to do something with it, now.

My next goal is to create a user authentication. After a lot of thoughts on if/yes/no I decided to use the FOSUserBundle – hoping it will provide some features I could use and still would integrate if I want to use OPTIONAL ALTERNATIVE Facebook and Twitter login…

Step 7: Add FOSUserBundle

I’m following the steps in the docs and add the line to composer and do an update. It suggests Doctrine, BTW, but I ignore that, thank you. But, for Propel, it says I need to move my schema.xml to app/Resources/FOSUserBundle/config/propel/schema.xml. Well, my schema already contains a User (which is unused atm) that contains an id – so it might work?!
Now did the configurationa steps of the doc (including installation of the typehintable behaviour) and reran my propel-build… SUCCESSFULLY, so I really could see a page on /app_dev.php/login. Impressive!

Well, no, it did not… First: I missed that I should copy the schema.xml of the bundle to “app/……”. Second: Looks like I got mislead with my expectations about schema.xml, where to put it, and where Models will be generated… Can I have FOSBundle generate it’s User class into my Bundles folders? How can I create relations between the User Class of FOSBundle and my app?

Questions to be answered later, I hope

Symfony2 and me – let’s be friends – Part 2

I have a running, blank symfony2 project now, with Propel instead of Doctrine – but no DB or bundle yet

Step 5: Gimme a bundle

I know there is this bundle generator, so let’s see it:
app/console generate:bundle
I chose my namespace and hit some [Return]s – and I’m good to go – WOW, yes it’s there! Not much in there, yet, but I’ll check later.

Step 6: The database

Nice so far! Now I’d like to connect to my exisitng database and use Propel’s magic database reverse-engineering
Thanks to composer I don’t have to deal with namespace and prefix configurations, so I can go to the db- connection settings directly
So I add the basic propel config with parameters (http://www.propelorm.org/cookbook/symfony2/working-with-symfony2.html) to my app/config.yml; then I need to set the paramters in parameters.yml
No, how can I check if the connection works? Why not run the reverse-engineering process directly?
app/console propel:reverse “No generated files” Hmmm, my connection might not work – but there is no error 🙁
SO, I manually check the permissions again (quickly dumping an adminer into the web dir) and YES, there is a problem with the permissions. SO I fix this, then run app/console propel:reverse again – and I have a file!!!

Step 7: Adjust the db schema

For the time being, the schema looks quite ok, so I’ll just move it from the generated-schemas folder to a more appropriate destination, being the Resources/config folder of my bundle.
To run propel:build now, I only need to add a namespace=”my\bundle\namespace” to the database tag of the xml, and it generates my models!

Enough preparational stuff, more coding next days!

Symfony2 and me – let’s be friends – Part 1

My journey, starting a rewrite of an exisitng project from scratch with Symfony2

Step 1: Get the framework:

composer.phar create-project symfony/framework-standard-edition path
After giving my VM some more RAM (512MB wasn’t enough) it succeeded.

Step 2: Get rid of Doctrine and grab Propel

So I edit the composer.json file and delete the two doctrine lines (if someone other lib needs doctrine/common for annotations, they will solve the dependency, right?
Then add "propel/propel-bundle": "1.1.*" to the file instead, as stated on http://www.propelorm.org/cookbook/symfony2/working-with-symfony2.html
Then I run “composer.phar update” and it updates my monolog, removes doctrine and installs propel, its bundle and phing.
After that, however, the symfony cache update fails – if course, I removed Doctrine which is configured somewhere in the the Kernel.

So I remove the Doctrine line from app/AppKernel.php and all doctrine stuff from app/config/config.yml – and HOORAY, my app/console is green again

Step 3: Does it work?

My webserver is already configured to check for an app.php in the web subfolder, so it works – but first I’ll add my IP to app_dev.php so I can see all the debug beauty of Symfony2
Checking the Debug toolbar I notice Propel is missing – of course, I need to add it to the kernel where I kicked out Doctrine
So I add new Propel\PropelBundle\PropelBundle(), to app/AppKernel.php where I commented Doctrine befor, and app/console knows some good old Propel commands now.

Step 4: Freeze this into git

This is a good start, so I want to get this status into git. I’m surprised to see that there IS already a lot of git-information in the directory, being the symfony2 history.
Quick fix, rm -rf .git and git init again, here I come. On the plus side, I already have a nice .gitignore 😀
git add .; git commit -m “Fresh start”

Symfony Day 2011 in Cologne

Symfony Day was amazing!!!

I love the community and it was great fun to see all the guys I already met during the last years and this year’s Symfony Live in Paris.

Again, a big THANK YOU to all organisers and of course all the people that make Symfony2 happen!

I just wanted to quickly paste my (sparse) notes that I took for myself – in case they help out someone else as well.

Basically, they are just some reminders for me of stuff that I wanted to check out.

Silex

  • Silex routes can use value() and assert() to be enhanced
  • Silex needs cookbook contributions
  • Silex can easily be functional tested (and require “” can have a return value!)

SPL

A shame, I never really looked at it before!? So take a closer look at http://php.net/manual/de/spl.exceptions.php and SPL in general

Links to follow/read

Pragmatic Programmer: http://pragprog.com/the-pragmatic-programmer/extracts/tips

Sonata

Check out www.Sonata-project.org – adminbundle

See, if it’s really that complicated as the talk made it seem

I did not note so much else, either because I already knew it, wanted to take a look at the presentations later, or player #sfdayjeopardy (thanks to Jeremy “Rick Astley” Mikola for that)

There’s one more thing

Actually, there were two things: Fabpot is a visionary! And his vision, of looking at Symfony2 not (only) as a framework but as a collection of several, individual and powerful components is great. MAYBE not for the developer looking for “a thing that does all the magik™” but for the more experienced one that can pick the best of a wide variety of components. Like some other PHP projects do now…

AND: Me, being a big fan of Propel, was happy to see all the momentum propulsion it got recently, including development support from inside sensio.

Hope to see you all again in June(?) in Paris and in November(?) in Berlin?

My biased view on Doctrine vs Propel

I’m getting old and I’m old fashioned.

I’m still using mysql, not mongoDB.

I’m using php and apache, not node.js, ruby, python or lighttpd, nginx or ….

And I’m using propel again (1.5), not doctrine. AND I’M LOVING IT!!

After #sfdaycng 2009, the presentations and talks there, I forced myself to giving Doctrine a try and really liked it then. A lot of reading through documentation in the beginning if you are new to it, but hey, that’s normal. However, today, even when writing a simple query, I still have to consult the docs. Why? The syntax seems not to be “my style” somehow, it won’t fit into my type of writing code.

But there is such a HUGE difference between Doctrine and Propel and Doctrine is SO MUCH BETTER

Is it? Do I care what’s inside if I’m only querying a database without using all possibilities? Remember sfLive 2010?

“Is Doctrine re-inventing propel” Hahaha LOL ROFL LMAO

At this years big symfony conference #sflive2010, in a Q&A session about Doctrine2, a poor guy obviously asked the question, that made it into a big running gag: “Are you trying to re-invent propel?”

I have to admit and regret that I was not there, but if you followed the #sflive2010 hashtag on twitter, it was not very long until this got tweeted over and over again. And everybody was making fun of it…

Sure, it is kind of funny to ask @jwage, (lead) developer of Doctrine that question… But hey? What’s really so funny about it?

While everybody in the audience laughed, it really would have been interesting to

a) see how many were trying to smirk, wondering whats so funny about that

b) know how many would have been able to give a good reply on that?

And with “good reply” I mean, explain the difference.

And with “difference” I mean, explain it beyond

  • “doctrine has the schema.yml and the model classes in a subfolder”
  • “propel uses two classes User and UserPeer, doctrine has User and UserTable”
  • “propel has a criteria object, doctrine a query object”

If you started with symfony 1.0 as a developer that did not work with frameworks like symfony before (like I did), everthing is “symfony” – may it be propel, phing, creole, PDO, … and getting into these differences was not really worth it in the first place if you wanted to get started with writing your first applications.

What I’m trying to say: I guess that a lot of symfony developers out there don’t know or care about “Doctrine” vs “Propel” – they use the default. And they call it “symfony”, thinking that it belongs to it. If they started with symfony <1.2, they’ll use propel, now they use doctrine – because it’s the default. You learn it, you use it, you stick to it.

And I sticked to propel throughout 1.2. And it was quite save for me getting along ignoring Doctrine. Just then, when I started using Doctrine after #sfdaycgn I found new things, new approaches, things I liked, things I had to learn and a lot of stuff I could not get the hang of.

So now, with propel 1.5 and the new Query objects, I gave it a try and was so excited, I decided to come back to Propel. Just because it “feels like home” and fits my personal style of thinking/coding.

And of course both ORMs have their features and behaviours and at #sfdaycng Doctrine had more of them and so on… But I also assume that a lot of those features are beyond most everyday developer’s needs

But Propel is dead

Oh no, it’s not!!!

With the new 1.5 it REALLY got amazingly easy to write your code, extend your Query Model and easy query creation. 1.6 is on it’s way and 2.0 was just recently announced to be based on Doctrine2, as a layer on top, providing the usability features… that’s something we will have to wait for, but I does sound very good.

But Benchmarks show that…

So what? I couldn’t care less. Why should I? I am not running sites that have gazillions of users with quadrubillions of tables and hexafoobarillions of rows – I don’t care if grabbing all posts with a certain topic and tag for a given user takes 111 ms or 132 ms. But I do care if I have to consult the docs for writing that query because I couldn’t remember the syntax or – on the other hand – my IDE (yeah, right, eclipse, I’m REALLY old fashioned) nearly automagically creates the query for me.

But what are you actually trying to say?

  • Most developers don’t need benchmarks to find their ORM – it’s about own style and preference (or customer specs, but thats another story)
  • Some symfony developers might not even have thought about giving “the other one” a try.
  • Personally, I get along with propel 1.5 better
  • I’m getting old

So, if you know Doctrine, give Propel 1.5 a try.

If you only use Propel, give Doctrine a try.

And if you still use Propel 1.4 – MAN!!! Get the 1.5!!!

Any comments appreciated, I really like to see what I’m missing

PS: I’m even more old fashioned: I’m using vi – not vim, but eclipse with the vi plugin. Why? REAL code completion! A wonderful thing with propel 1.5!

Starting to get addicted to unit testing…

During the last months, I started paying more attention to unit testing and writing tests in my symfony apps. But not until this awesome post from Bernhard Schussek, I really understood how you can make your testing life easier and get tests done better, faster and be more efficient.

It also helped me to understand, to which granularity you can and should break down your tests as well as your methods in the lib classes.

So this really is a BIG FAT recommendation on reading the article!!

What I learned from it and what I am doing ad enjoying right now, when creating new functionality:

  • If I have a complex method to write, I break it down into the smaller parts that need to be done step by step. For this, I write the logic with methods that are not even exisiting yet.
  • Next, I write the unit TESTS for these non-exisiting methods, filled with test cases that make sense in the current context of the problem and help to make sure the method works
  • Next, create an empty stub of the new method, so you can start running the test without a crash – but of course it will fail
  • Add functionality to the method to make sure you finally pass all tests.

This really makes the whole process a lot of fun because it really is rewarding to see the “All tests successful.” or “# Looks like everything went fine.” at the end.

And you can continue with your next part of the more complex functions with a good feeling of confidence, that in the end it will all work fine 😀

Planned something new: KaroAPI

Planning, planning, planning…
Last night I read some stuff about RESTful applications… and of course, quickly, the following plan came to my mind: I need it! Somewhere!
And of course, as always, the best baby for testing and applying new techniques is www.karopapier.de
So I decided I need a KaroAPI, using RESTful approaches.

Digging through it, the next question came: Should I use XML or JSON for the output? Or both?

JSON has the advantage to be shorter, I like it better AND: It could be used in a new JavaScript based frontend. Imagine to simply to a AJAX/JSON request for the details of a game and nicely format the data in a Browser.

XML, older, more widely spread, is more commonly used and maybe can be parsed with some clients more easily – and applying XLS/XLST it could also be readable and nicely formatted for humans.

Next thought was: “Well – offer both” – creating an object with PHP and then either apply a json_encode or XML_Serialize should not be a big deal, right?
Well, we would see…

And not enough of all the big plans, I thought I could give symf Symfony2 a try.

Sooooo:
Big plan, draft 0.3:

  • KaroAPI
  • JSON and XML representation of data (for bots, clients and other KaroTools)
  • use Symfony2 for the implementation

Added to the todo list… the big problem – already so much on this list. And I never manage to do it one by one, but get distracted by so many other interesting things…
Like, lately, when I introduced APE into the chat. But, Hey, I did not even blog on that yet…

Süß, so ein d1d.li

So, heute hat man nicht nur ein Blog und einen Twitter-Account, sondern auch seinen eigenen URL-Shortener. tinyurl war früher mal DAS non-plus-ultra, aber seit twitter zu noch mehr Zeicheneinsparung zwingt, sprießen sie wie Pilze aus dem Boden, die bit.lys, digs.bys und sonstige Kurzmacher…

Naja, und dann wollt ich das auch haben – und daher gibt es jetzt http://d1d.li. Ist nicht öffentlich – und verweist von sich aus eigentlich wieder hierher, aber ich werde es für mich selbst als Link-Liste nutzen, um merkenswerte Links abzulegen.

Was? Bookmarks im Browser? Ach, quatsch… out, völlig out!!

Und das ganze läuft natürlich auf symfony (und sqlite) und war eigentlich, dank des schönen object-routings, ganz schnell erledigt.

PS: Ich dachte, die Steuersünder aus Liechtenstein zu unterstützen ist das kleinere Übel als die Terroristen in Libyen, des wegen li statt ly.