This is the content for my Agile2012 presentation. (Download BigDataLittleTests-Heintz)
Slides are attached to this post, and these are the GitHub repos:
- https://github.com/jheintz/sample-hadoop-testing
- https://github.com/jheintz/sample-riak-testing
- https://github.com/jheintz/riak-download
Here are the scripts that demonstrate everything:
Hadoop Sample:
#/bin/sh
git clone https://github.com/jheintz/sample-hadoop-testing.git
cd sample-hadoop-testing.gitgit checkout step1 # initial MRUnit tests
mvn test
# eclipse: show mapper
# eclipse: show reducer
# eclipse: show unit test
# eclipse: try open hadoop source…git checkout step2 # Use Cloudera repo for sources
mvn test
# eclipse show pom
# eclipse: open hadoop sourcegit checkout step3 # parameterized many tests
# eclipse test ManyWordCountTest.javagit checkout step4 # parameterized non-mr tests for speed
# eclipse test ManyWordTestgit checkout step5 # add cluster test
# eclipse show HadoopClusterBase
# eclipse test ClusterTestgit checkout step6 # add parameterized cluster test
# eclipse test MultipleClusterTestgit checkout step7 # split unit and integration tests
# eclipse show test names
# eclipse show pom failsafe
mvn test
mvn verify
Riak Sample:
#/bin/sh
git clone https://github.com/jheintz/sample-riak-testing.git
cd sample-riak-testing.gitgit checkout step1 # initial test, fails
mvn test
# eclipse show SmokeTest failuregit checkout step2 # add riak downloader
mvn test
# I don’t know why eclipse has maven failure…
# eclipse show riak-build.xml
# eclipse show pom.xmlgit checkout step3 # add parameterized many tests
# eclipse show ManySmokeTestant -f riak-build.xml stop
Cassandra Sample:
#/bin/sh
git clone http://github.com/jsevellec/cassandra-unit.git
cd cassandra-unitmvn test
mvn eclipse:eclipse
4 thoughts on “Big Data, Little Tests”
I was unable to download attached slides. Can you please email them to me. I attended your session at Agile 2012
Vinayak
Thanks for attending my presentation Vinayak, I’ve fixed the download now and will email you a copy!
What did you think of the session? What did you want more of? Less of?
Hi John,
Great Work. Really appreciate this a very useful contribution.
Do you have any recorded material of you presentation? If yes can you please give me some download location of it.
As I have not attended your session, so can you please help me out if possible.
Thanks,
Amar
Hello Amar
I do not have a recording of the session, but the git repo I listed in the blog post is actually a step-by-step walk through.
Let me know if you have specific questions or need more support with the materials.
Cheers
John