DevOps for Drupal: Why We Cook With Chef

DevOps for Drupal: Why We Cook With Chef

• 1.
  DevOps for Drupal:Why We Cook with Chef Wednesday May 22nd DevOps Track Beginner
• 2.
  Overview: Purpose Why arewe all here? •  Interested in DevOps practices •  Interested in automation tools like Chef •  Interested in using them with Drupal +
• 3.
  Overview: Sharing How arewe going to do this? •  Team style presentation and discussion with views from both sides - Development and Operations •  Share our experiences and examples... we will be keeping it real •  We are NOT doing a tool comparison +
• 4.
  Overview: Learn What dowe hope you will learn? •  Why DevOps matters •  Why automation is a must •  Why you can use Chef to help yourselves, make life easier, and save time +
• 5.
  For more infoon Chef, check out fellow DevOps track session: The Joy of Cooking - Whip Up a Drupal Environment with Chef •  Thursday May 23rd at 2:15pm •  OR 203 •  Presented by Opscode's Nathen Harvey Overview: More on Chef +
• 6.
  Who We Are Asoftware development shop... •  specializing in custom Drupal development, systems integration, mobile, DevOps, and 24x7 support •  based in Chicago with 30+ team members worldwide ...and yes we are hiring! Come talk to us! +
• 7.
  Who We Are Playingthe part of: Moderator and Product Owner @ Promet Source: Director of Products jay@promethost.com + Jay Uhlinger
• 8.
  Who We Are Playingthe part of: Development @ Promet Source: Solutions Architect will@promethost.com drupal.org: wamilton twitter/hub: @winmillwill + Will Milton
• 9.
  Who We Are Playingthe part of: Operations @ Promet Source: Systems Administrator greg@promethost.com + Greg Palmier
• 10.
  Audience Quiz Time Howmany are Developers? (a Will) + CC Image courtesy of Pedro Lozano on Flickr
• 11.
  Audience Quiz Time Howmany are Sysadmins? (a Greg) + CC Image courtesy of Sharyn Morrow on Flickr
• 12.
  Audience Quiz Time Howmany are both? (changing Prod servers) + CC Image courtesy of Arthur Caranta on Flickr
• 13.
  Audience Quiz Time Howmany are other? (the Product Manager) +
• 14.
  Audience Quiz Time Howmany are using automated configuration management tools and processes now in your everyday work with Drupal? How many are using Chef? +
• 15.
  DevOps: What ItIs Key Culture •  Foster a collaborative working relationship between Development and Operations Key Concept •  Infrastructure as code Key Results •  Higher deployment rates •  Better reliability, stability, and resilience of Production +
• 16.
  DevOps: 3 KeyPatterns 1. Make environments available early in the Development process 2. Shorten and amplify feedback loops 3. Create reusable deployment procedures +
• 17.
  DevOps: What ItIsn't It is not a collection of bash, Drush, insert scripts here, etc. + DevOps is not about the tools... they don't fix everything magically But DevOps relies on some tools to implement its principles
• 18.
  DevOps: Why IsIt Important One-off environments and complicated manual builds and deployments = FAILURE Must automate to meet business demands -- support agile, lean, faster, better... Every environment is Production to someone Drupal is not different -- all of these apply +
• 19.
  A Classic Scenario Jay:Will, you said this was working. Will: It works in Dev. Jay: Greg, can you look into why it isn't working in Prod? Greg: Ok. (later on) @#$%! Dev and Prod are not the same! +
• 20.
  Another Classic One TheBlog Post: Installing Drupal on AWS 1. Start wizard 2. 50+ steps later 3. A Drupal install +
• 21.
  Why We ChoseChef The orange color matches ours of course! We already had experience (Marius) doing DevOps work with Chef in Bay Area with leading tech companies We build Drupal applications -- we found Chef to be a good fit for application configuration management +
• 22.
  The Operations part... ...outwith the old and in with the new +
• 23.
  Our Operations Story Howwe started with Chef at Promet Inconsistency was killing our time How realistic was automation and configuration management of all of our stuff +
• 24.
  Wasting Time You arenot in the sudoers file. This incident will be reported. $ which git $ Fatal error: Class 'PDO' not found in blahblahblah All of these messages diverted your work +
• 25.
  Manual Config RunAmok •  Dev environments != Prod environments •  Shared Dev/Prod/?? environments were continually bloated •  Over time modifications amounted to more inconsistency between environments •  Documentation was lacking o  No records explaining changes o  Constant paper trail "paving" o  No great place for it anyways (what wiki?!?) •  More customizations -> High Fragility +
• 26.
  Typing Faster !=Solution •  Runbooks, Bash and SSH o  Process to stand up new infrastructure was done manually via a "runbook” •  Any configuration time was repeated extensively •  Environment “discovery” bled time o  Snooping around to analyze how things were initially stood up and potentially modified (caching, etc.) •  Endless SSH-ing into servers +
• 27.
  Chef Migration •  Inwith the New o  2 dedicated servers §  VM hosts o  Similar model, consolidated o  Entirely Chef managed o  Individual client Chef-spun Dev instances •  Stand Up, QA, and Cutover o  Virtualize legacy o  Considerations mainly PHP •  Monitoring customizations •  Special clients o  Non-Drupal o  Contractually unique +
• 28.
  Chef Migration And migratingwasn't a complete nightmare.... +
• 29.
  Righting the Ship • Server configuration in Git o  Mitigated "paper trail" o  Team awareness of configs by notifications o  PHP, Apache, MySQL •  System config changes pushed to Chef for Server propagation o  No more endless SSH-ing o  No more runbook config for every server o  Undoes "helpful" customizations by others •  Direct between me and Config Mgmt o  Configuration entirely “local” with Vagrant o  Git driven accountability and awareness (--stat) o  `knife` is to Chef as drush is to Drupal +
• 30.
  Client Ownership •  Clientsmaintain ownership of their assets o  Any platform with a knife plugin o  Legacy apps migrated o  Non-Drupal or Promet "old" clients •  Give us their Keys o  Let us be the Designated Driver o  Developers drop in, get pitted +
• 31.
  Client Ownership •  Excessivecustomization (not perfect) o  Non-Drupal clients or other work o  More so, use client's Hosted Chef •  Contract flexibility o  No Dysfunctional Marriage •  Honor prior client / hosting relationships o  Help clients where they are +
• 32.
  Things That AreSolved •  User Configs o  .gitconfig o  .tmux.conf o  .ssh/config o  shell customizations o  Not necessary but....doable •  Server Configs o  logwatch o  mail o  mysql o  apache o  ssh o  automysqlbackup +
• 33.
  Sysadmin Transformation Have timeto explore/make new tools Do support in a consistent fashion Automate all the Things! "Grow" your team and infrastructure not with numbers but with talent We do more work with less resources! +
• 34.
  Sysadmin Team Evolution Weeven spun-off a Startup company for infrastructure and applications automation and scaling using Chef in the Bay Area (non- Drupal) I don't care what you automate with... pick one and do it! +
• 35.
  The Dev part... ...it'snot what you think +
• 36.
  What I ThoughtI Would Say •  What it was like... •  How it sucked for Devs •  How we changed it •  How it sucked in new ways •  How that lead to the new thing •  Repeat.... •  Where we are now •  What's really fun about this for Devs +
• 37.
  The Problem •  Wewent through too much shit •  It's not even fun to talk about •  It's probably boring •  Someone else already said what I was trying to justify •  You're probably the choir if you're here +
• 38.
  Drupal: OH @crell "Ifyou're not using Features, you are not doing Drupal professionally" Also: •  If you don't have a one step build, you aren't using Features correctly => Sucking at doing Drupal professionally •  Configuration module is also probably fine •  It's going into core +
• 39.
  So Now MyPresentation Is •  What we're doing now •  Why you shouldn't NOT do what we're doing •  Actual examples of how it has helped •  Why Chef is really awesome •  A surprise +
• 40.
  What We're DoingNow •  Everything you need in the repo o  Any scripts or scripting examples o  README files o  (small(-ish)) csv's/xml/etc o  ...pretty much any text file that is not too large and is useful •  ...Still need a way to automate sourcing archives of files and sql... •  Vagrantfile + Berksfile +
• 41.
  Berks What? +
• 42.
  +
• 43.
  Kirkberks? +
• 44.
  What a DevHas to Do on Our Team •  Install git somehow •  Install the vagrant package •  Install the vagrant-berkshelf plugin •  git clone/pull && cd-to-the-root •  vagrant up (or reload) •  edit the hosts file or use dnsmasq or similar +
• 45.
  Some Concrete Examplesof This Helping Us Do Better •  Audits o  http://reload.github.io/phing-drupal-template o  Not perfect... o  But now no one has to repeat what is automated •  Every single deployment ever since o  Your Production push should not be the first time you attempt sync'ing to a Production environment o  ...unless you crave excitement •  Last minute accommodations o  Ever test RewriteBase? +
• 46.
  Add the .htaccess directive...+ ...change the docroot in code!
• 47.
  Potential Examples: You Can'tProve the Negative •  PECL/Pear packages o  Shame on professionals who can't figure it out... o  ...but shame on me for betting the project •  Anything to do with email ever since ever •  Anything to do with external services o  ...memcached o  ...SOLR o  ...even MySQL +
• 48.
  Why You Shouldn'tNOT Do As We Do Challenges you can deal with, excuses that are bullshit On stupid things people say re: vagrant and chef •  We make great docs! •  My native environment is fine •  Virtualization consumes resources •  I don't have enough time +
• 49.
  +
• 50.
  My Native Environmentis Fine 1. No, it isn't •  Your job is to ship to your downstream o  ...not manicure and fetishize your tools •  You are getting strictly less help than you could get o  ...even if you're getting a lot of help o  ...even if you're paying for help •  You are keeping yourself from helping +
• 51.
  My Native Environmentis Fine (cont'd) 2.Your native environment is personal •  You may even obsess over it •  You may be completely wrong •  You will be happier without each other +
• 52.
  Virtualization Consumes Resources +
• 53.
  I Don't HaveTime... ...to do everything perfectly •  We didn't either, and we're doing far better than we were •  We do not have a massive continuous delivery, cloud orchestration and scaling infrastructure •  ... But now what we have is in code +
• 54.
  Chef is ReallyAwesome ...Puppet is probably cool too •  Stewardship and leadership •  Constantly evolving ecosystem •  A community hell-bent on sharing •  Kicks ass even if you don't +
• 55.
  So Easy ICan Share It with You! Don't try it on this connection bash <(curl -L -s http://bit.ly/10McZSZ) github.com/promet/drupal_cookbook Get to work already! +
• 56.
  Why You ShouldUse this Project (or something like it) We must advance the state of the art for Drupal deployment and application management The CMS that is easiest to flexibly deploy will be the CMS of the future All the features and D8 config exports in the world can't help you if you can't automate it +
• 57.
  Other Projects/Strategies •  Ifyou have a dope Jenkins workflow... o  help me automate it with Chef so we can all use it! •  Please use Puppet if it suits you o  and be vocal about hardcoded conventions that make your work hard •  Kraftwagen - kraftwagen.org •  Ariadne by patcon - github.com/ myplanetdigital/vagrant-ariadne +
• 58.
  The Wrap-Up part... ...whatshould you remember +
• 59.
  Key Summary Points Processand team is the focus •  Get your Devs and Sysadmins in the same room and start communicating Tools are there to help •  Start automating now Chef is an awesome tool •  It works and it helps +
• 60.
  Q&A Ask us questionsnow Talk to us afterward Stop by and see us at Booth 201 Thanks! +
• 61.
  Feedback? Evaluate session at: portland2013.drupal.org/schedule