Skip to main content
UploadLanguage (EN)Support
BusinessMobileSocial MediaMarketingTechnologyArt & PhotosCareerDesignEducationPresentations & Public SpeakingGovernment & NonprofitHealthcareInternetLawLeadership & ManagementAutomotiveEngineeringSoftwareRecruiting & HRRetailSalesServicesScienceSmall Business & EntrepreneurshipFoodEnvironmentEconomy & FinanceData & AnalyticsInvestor RelationsSportsSpiritualNews & PoliticsTravelSelf ImprovementReal EstateEntertainment & HumorHealth & MedicineDevices & HardwareLifestyle
Jon Jagger, profile picture
Uploaded byJon Jagger
PDF, PPTX2,672 views

Design and Evolution of cyber-dojo

Jon Jagger, a software consultant, shares the evolution and design of Cyber-Dojo, a cloud-based interactive coding environment aimed at enhancing coding dojo experiences. Initially inspired by a 2009 coding event, he began developing Cyber-Dojo in 2010, which allows users to practice coding anonymously and efficiently using the Docker technology for testing. With over 8888 commits by 2016 and a focus on educational use, Cyber-Dojo emphasizes both individual and team-based learning in coding markets.

Softwareā—¦
Download as PDF, PPTX
twitter : @JonJagger email : jon@jaggersoft.com blog : http://jonjagger.blogspot.co.uk/ charity : http://cyber-dojo.org the design and evolution of cyber-dojo Hi. I'm Jon Jagger, a self-employed software consultant. I love ļ¬‚y ļ¬shing for salmon ļ¬shing. If you hire me and you're near a salmon river I'll reduce my rate!
late 2009(about tea-time) Late in 2009 I was doing some consultancy in Oslo.
One evening I attended a code dojo being held in the Scotsman pub.
In case you're not familiar with the word, Dojo is the Japanese name for the place where martial artists meet to practice their martial art. So a code dojo is where coders meet to practice coding. Uncle Bob was the guest of honour. He selected the battleships coding exercise and also speciļ¬ed the language (I think it was Ruby) and the test framework. There were about 30 developers - split into 6 teams of 5 people. We then had about 90 mins to code, after which each team would connect their laptop to a projector and show their work. At least, that was the plan.
Somewhat predictably, most of the groups struggled to connect their laptop to the projector. Quite a lot of time was wasted.
Two groups didn't have much to show as they had spent more than half the time struggling to install the language and test-framework Uncle Bob speciļ¬ed. But eventually everyone had shown their work and Uncle Bob pronounced one group the winners. Which was a bit strange.
In my hotel room that night I was thinking about the wasted time. Suppose everyone had been working in a shared cloud-based environment? There would have been nothing to install. Only Bob's laptop would need connecting to the projector. The more I thought about this idea the more I liked it.
At some points in your life there are times where you decide something is worth spending your time on. This was a moment for me. I decided to build an IDE, an Interactive Dojo Environment. I rolled my sleeves up and started learning lots of new stuļ¬€ I'd need. I didn't know any Ruby, HTML, CSS, Rails, Javascript, etc.
https://github.com/cyber-dojo/web It's called it cyber-dojo. It's open-sourced on github. The ļ¬rst commit was in January 2010. It's now 2016 and 8888 commits later. Things don't come into existence instantly!
DEMO Me demoing USING cyber-dojo.
Cyber-dojo is completely free for non-commercial use. For commercial use you need to buy a license.
http://blog.cyber-dojo.org/2015/08/cyber-dojo-foundation.html All license fees go to a registered charity (cyber-dojo Foundation) which buys Raspberry Pi computers for kids.
40,000+ practice-sessions Of course, even if you're non-commercial, your welcome to make a donation too :-)
VALUE SYSTEM? That's a bit about its evolution. Now a bit about its design...
Interactive Dojo Environment Integrated Development Environment ā€¢ make money ā€¢ individual based ā€¢ finish focused ā€¢ test driven? ā€¢ work faster ā€¢ time pressure ā€¢ don't repeat ā€¢ have fun ā€¢ team based ā€¢ learning focused ā€¢ test driven ā€¢ work slower ā€¢ no time pressure ā€¢ repeat It was clear to me from the beginning that a cloud-based environment designed for coding dojos should be diļ¬€erent to a regular development environment....
ā€¢ team based ā€¢ individual based ā€¢ no login ā€¢ anonymity No login. No username, password. Anonymity is very important. When reviewing we cannot talk about Bert and Ernie's code. We can only talk about the Lion's code. Bert and Ernie can decide whether or not they want to reveal that they are the Lion.
ā€¢ learning focused ā€¢ finish focused I have come to realise that developers derive enormous value from simply having their steps replayed to them. For example this is a real example from some consulting I did recently in Bangalore. From the burst of green traļ¬ƒc lights it looks like the koala is refactoring. They are refactoring the duplication out of their tests. Nice. Although, as another developer pointed out, the use of the word if in the assert helper is a bit dubious.
tests ran one or more failed tests ran all passed tests did not run I extend the standard red/green fail/pass colour scheme. I complete the traļ¬ƒc-light by using amber for cases when the tests do not run (eg syntax error)
{ "visible_filenames": [ "README", "Hiker.feature", "Hiker.java", "HikerStepDef.java", "RunCukesTest.java", "cucumber.xml", "cyber-dojo.sh" ], "filename_extension" : ".java", "display_name": "Java, Cucumber-Spring", "image_name": "cyberdojofoundation/java_cucumber_spring", "red_amber_green" : [ "lambda { |output|", " return :red if /FAILURES!!!/.match(output)", " return :green if /OK ((d*) tests)/.match(output)", " return :amber", "}" ] } .../Java/Cucumber-Sprint/manifest.json Determining the red/amber/green static happens in a simple function held in the language+test_framework's manifest. It's typically implemented using simple regular expression matchers.
One of the things cyber-dojo is very eļ¬€ective at (which I had not anticipated) is how well a short 45 minute practice session can mimic problems normally associated with large projects typically running for months and years. For example, waterfall development, when stuļ¬€ is endlessly 90% done and overruns. In the above dashboard (from the same trip to Bangalore) note that the two animals with the fewest traļ¬ƒc lights are the frog and the snake. Note the time-gap (left to right) before their ļ¬rst visible traļ¬ƒc-light, which, almost inevitably, is amber. Note too the burst of ambers following this initial amber as they try to "recover". The frogs just got to green in time (lucky) but the snakes did not (very typical). Again, inevitably, the snakes ask for a little bit more time! They're almost done, they say. But no. There is no more time. We are not shipping anything!
Reminder to drink so as not to lose my voice.
How does (and did) it work under the hood?
2010 - No security! (frequent backups) In cyber-dojo arbitrary code is written in the browser and run on the server!!!! This has obvious security implications. Yet the ļ¬rst version had no security. Really! It was the simplest thing that could possibly work. I had mostly built cyber-dojo for myself. When I used cyber-dojo during paid consultancy I would simply spin up my own personal server.
Initially it was just a bare server with rails installed.
.py Python pytest apt-get update && apt-get install --yes libsqlite3-dev sqlite3 bzip2 libbz2-dev wget wget http://www.python.org/ftp/python/3.3.5/ Python-3.3.5.tar.xz tar xJf ./Python-3.3.5.tar.xz cd ./Python-3.3.5 && ./configure --prefix=/opt/ python3.3 cd ./Python-3.3.5 && make cd ./Python-3.3.5 && make install ln -s /opt/python3.3/bin/python3.3 /bin/python apt-get update && apt-get install -y python-pip pip install -U pytest Python pytest It really was just the server. For example, to support Python using pytest (on the browser), then Python and pytest had to be installed on the server.
.py .py Python pytest Python pytest When the dev hit the test button on the browser the source ļ¬les were copied on to the hard-disk of the server and run directly on the server! There was no security. No chroot. Nothing!
$ cd / $ rm -rf * 2013 This worked ļ¬ne for several years on the public cyber-dojo.org server until one day someone ran the above code! They sent me a nice email saying how sorry they were, that they had just assumed it wouldn't work!! It was an ex server.
0.3 I looked for ways to provide security. A new, little known (at that time) technology called Docker caught my eye. It was at about 0.3 then. Docker went open source in March of 2013.
FROM cyberdojofoundation/build-essential MAINTAINER Jon Jagger <jon@jaggersoft.com> RUN apt-get update && apt-get install --yes libsqlite3-dev sqlite3 bzip2 libbz2-dev wget RUN wget http://www.python.org/ftp/python/3.3.5/Python-3.3.5.tar.xz RUN tar xJf ./Python-3.3.5.tar.xz RUN cd ./Python-3.3.5 && ./configure --prefix=/opt/python3.3 RUN cd ./Python-3.3.5 && make RUN cd ./Python-3.3.5 && make install RUN ln -s /opt/python3.3/bin/python3.3 /bin/python RUN apt-get update && apt-get install -y python-pip RUN pip install -U pytest Dockerfile Now the commands to install Python and pytest go into a ļ¬le called Dockerļ¬le.
cyberdojo/python_pytest $ docker build -t cyberdojo/python_pytest . FROM cyberdojofoundation/build- essential MAINTAINER Jon Jagger <jon@jaggersoft.com> RUN apt-get update && apt-get install --yes libsqlite3-dev sqlite3 bzip2 libbz2-dev wget RUN wget http://www.python.org/ftp/python/ 3.3.5/Python-3.3.5.tar.xz RUN tar xJf ./Python-3.3.5.tar.xz RUN cd ./Python-3.3.5 && ./configure -- prefix=/opt/python3.3 RUN cd ./Python-3.3.5 && make RUN cd ./Python-3.3.5 && make install RUN ln -s /opt/python3.3/bin/python3.3 /bin/ python RUN apt-get update && apt-get install -y python-pip RUN pip install -U pytest Dockerfile Then, on a system with docker installed, you build your named docker image from the Dockerļ¬le. You've then got your image.
cyberdojo/python_pytest .py .py Python pytest Every time the dev presses the test button in the browser, the rails server spins up a docker-container from the appropriate docker-image (in this case the one for python +pytest), copies the ļ¬les into the container, and runs the test inside the container, grabs the output, and then destroys the container (but not the image). Creating a new container from an image is almost instantaneous!!
The images for the 50+ language&test_framework combinations oļ¬€ered on the setup page are all stored on the docker hub and can be pulled freely by anyone who has installed docker.
more animals! 2015 At ļ¬rst there were only 16 animals. Several people asked to be able to run practice sessions with a hundred or so participants and 16 was not enough. I needed more animal images! My friend Dymtro Mindra paid a professional artist (Nadya Sivers) to draw 64 new images. Aren't they great :-)
setting up your own server was lengthy and error-prone The biggest remaining problem was for people who wanted to run their own servers. Installing the rails server was lengthy and error prone.
cyberdojo/python_pytest .py cyberdojo/web .py Python pytest So I put the rails server into a docker image as well! (While I was at I put an nginx front end on too.)
cyberdojo/python_pytest .py cyberdojo/web:1.12.1 .py Python pytest 1.12.1 Eagle eyed readers will spot there is a problem putting the rails server inside a docker image. The rails server issues docker commands to run the tests submitted from the browser. But docker is installed on the server, not inside the image holding the rails server! So I also had to install docker inside the web image. This is sometimes called docker-in-docker. This creates a version dependency. The version of docker installed on the server (which spins up the web container) must match the version of docker installed inside the web container (which spins up the python_pytest container). Now installation is mostly installing docker and pulling the pre-built cyber-dojo docker images.
FROM cyberdojo/user-base MAINTAINER Jon Jagger <jon@jaggersoft.com> ... USER root RUN apk --update add ruby ruby-irb ruby-io-console ruby-bigdecimal tzdata bash ARG CYBER_DOJO_HOME USER root RUN mkdir -p ${CYBER_DOJO_HOME} COPY Gemfile ${CYBER_DOJO_HOME} RUN apk --update add --virtual build-dependencies build-base ruby-dev openssl-dev postgresql-dev libc-dev linux-headers && gem install bundler --no-ri --no-rdoc && cd ${CYBER_DOJO_HOME} ; bundle install --without development test && apk del build-dependencies ... Dockerfile cyberdojo/web:1.12.1 docker build Here's part of the Dockerļ¬le for the cyber-dojo web image. It shows the commands to install ruby and setup the rails server.
FROM cyberdojo/user-base MAINTAINER Jon Jagger <jon@jaggersoft.com> ... ARG DOCKER_VERSION USER root RUN apk --update add curl && curl -OL https://get.docker.com/builds/Linux/x86_64/ docker-${DOCKER_VERSION}.tgz && tar -xvzf docker-${DOCKER_VERSION}.tgz && mv docker/* /usr/bin/ && rmdir /docker && rm /docker-${DOCKER_VERSION}.tgz && apk del curl ... Dockerfile cyberdojo/web:1.12.1 docker build Here's part of the Dockerļ¬le for the cyber-dojo web image. It shows the commands to install docker!
A bit about testing.
#!/bin/sh set -e my_dir="$( cd "$( dirname "${0}" )" && pwd )" cd ${my_dir}/../docker/web-base ./build-image.sh cd ${my_dir}/../docker/web ./build-image.sh cd ${my_dir}/../cli ./cyber-dojo down ./cyber-dojo start-point ls --quiet | grep 'languages' && ./cyber-dojo start-point rm languages ./cyber-dojo start-point ls --quiet | grep 'exercises' && ./cyber-dojo start-point rm exercises ./cyber-dojo start-point ls --quiet | grep 'custom' && ./cyber-dojo start-point rm custom ./cyber-dojo start-point ls ./cyber-dojo start-point create languages --dir=./../../start-points-languages ./cyber-dojo start-point create exercises --dir=./../../start-points-exercises ./cyber-dojo start-point create custom --dir=./../../start-points-custom ./cyber-dojo start-point ls ./cyber-dojo up cid=`docker ps --all --quiet --filter "name=cyber-dojo-web"` docker exec ${cid} sh -c "cd test && ./run.sh" done=$? exit $done test/build_and_run.sh I have rails installed locally on my macbook. However, I do not run the tests locally. That would be simple, but it would be a trap. This is because the code being tested would not be running inside a docker container. Yet when the code really runs (after being deployed) it always runs inside a docker container. So instead I run a script which rebuilds the docker web image, starts a new web container, shells into this container, and runs the tests.
... env: global: - DOCKER_ENGINE_VERSION=1.12.1 - DOCKER_COMPOSE_VERSION=1.8.0 - DOCKER_DIR=${TRAVIS_BUILD_DIR}/docker before_install: # list docker-engine versions - apt-cache madison docker-engine # install specific version of docker-engine # install specific version of docker-compose ... .travis.yml github has an excellent feature whereby every commit can trigger a Travis continuous integration (CI) pipeline. I use this on the cyber-dojo web repo. The script github runs installs speciļ¬c versions of docker and docker-related tools...
... script: # Build the cyber-dojo web image - cd ${DOCKER_DIR}/web - ./build-image.sh ${DOCKER_ENGINE_VERSION} - cd ${TRAVIS_BUILD_DIR}/cli - ./cyber-dojo up - cid=`docker ps --all --quiet --filter "name=cyber-dojo-web"` - docker logs cyber-dojo-web - docker inspect -f {{.State.Running}} ${cid} - docker ps -a - docker pull cyberdojofoundation/gcc_assert:latest - docker pull cyberdojofoundation/ruby_test_unit:latest .travis.yml - docker exec ${cid} sh -c "cd test && ./run.sh" ...and then builds the web image, starts a new web container, shells into this container, and runs the tests. Just like when I run the tests myself :-)
Here's an example of the output you get from travis CI when the tests are run inside the docker container.
This is the dashboard for the Travis Continuous Integration. The various github repos are listed on the left. On the right are results from recent commits on the selected github repo (web).
thank yous Byran Wills-Heath Nadya Sivers Seb Rose Olve Maudal Dymtro Mindra Mike Long A big thank you also to these people who have all helped with cyber-dojo in various ways.
twitter : @JonJagger email : jon@jaggersoft.com blog : http://jonjagger.blogspot.co.uk/ charity : http://cyber-dojo.org that's all folks That's it. Thanks for listening. I hope I've sowed a few seeds. Have a great conference.