Rob Hirschfeld, CEO and Co-Founder, RackN talks with Stephen Spector, HPE Cloud Evangelist about the recent uptake in Ansible news as well as how Digital Rebar Provision assists Ansible users.
Listen to the 9 minute podcast here:
As this is the launch of L8ist Sh9y Podcast from RackN we encourage you to visit our site at https://soundcloud.com/user-410091210 or subscribe to the RSS Feed. We will also be publishing on iTunes as well shortly.
TL;DR: Operators need pay/status equity to succeed.
Charity Majors is one of my DevOps and SRE heroes* so it was great fun to be able to debate SRE with her at Gluecon this spring. Encouraged by Mike Maney to retell the story, we got to recapture our disagreement about “Is SRE is Good Term?” from the evening before.
While it’s hard to fully recapture with adult beverages, we were able to recreate the key points.
First, we both strongly agree that we need status and pay equity for operators. That part of the SRE message is essential regardless of the name of the department.
Then it get’s more nuanced. Charity, whose more of a Silicon Valley insider, believes that SRE is tainted by the “Google for Everyone” cargo cult. She has trouble separating the term SRE from the specific Google practices that helped define it.
As someone who simply commutes to Silicon Valley, I do not see that bias in the discussions I’ve been having. I do agree that companies that try to simply copy Google (or other unicorns) in every way is a failure pattern.
Charity: “I don’t want get paid to keep someone else’s shit site alive”
I think Google did a good job with the book by defining the term for a broad audience. Charity believes this signals that SRE means you are working for a big org. Charity suggested several better alternatives, Operations Engineer. At the end, the danger seems to be when Dev and Ops create silos instead of collaborating.
Consensus: Job Title? Who cares. The need to to make operations more respected and equal.
What did you think of the video? How is your team defining Operations titles and teams?
(*) yes, I’m working on an actual list – stay tuned.
TL;DR: Operators (DevOps & SREs) have a hard job, we need to make time and room for them to redefine their jobs in a much more productive way.
The Cloudcast.net by Brian Gracely and Aaron Delp brings deep experience and perspective into their discussions based on their impressive technology careers and understanding of the subject matter. Their podcasts go deep quickly with substantial questions that get to the heart of the issue. This was my third time on the show (previous notes).
In episode 301, we go deeply into the meaning and challenges for Site Reliability Engineering (SRE) functions. We also cover some popular technologies that are of general interest.
Author’s Note; For further information about SREs, listen to my discussion about “SRE vs DevOps vs Cloud Native” on the Datanauts podcast #89. (transcript pending)
Here are my notes from Cloudcast 301. with bold added for emphasis:
We’re very invested in talking about SRE and want to hear from you! How is your company transforming operations work to make it more sustainable, robust and human?We want to hear your stories and questions.
Welcome to the weekly post of the RackN blog recap of all things SRE. If you have any ideas for this recap or would like to include content please contact us at info@rackn.com or tweet Rob (@zehicle) or RackN (@rack ngo)
SRE Items of the Week
The Cloudcast #301 – SRE and Infrastructure Operations
http://www.thecloudcast.net/2017/06/the-cloudcast-301-sre-and.html
Description: Brian talks with Rob Hirschfeld (@zehicle, Founder/CEO of @RackN) about the concepts of SRE (Site Reliability Engineering), the challenges of maintaining infrastructure software, emerging tools and the next-generation of operations.
Show Notes:
DevOps Enterprise Summit London: Tales of courage and community
https://techbeacon.com/devops-enterprise-summit-london-tales-courage-community
After spending two amazing days with 700 of my closest DevOps cohorts from Europe, the Middle East, Africa, and beyond, I learned all about the latest and greatest IT and technology transformation reports at the DevOps Enterprise Summit London. With substantial growth in attendance from the first year, in 2016, the buzz around the show was palpable. And, what a location! From the venue, the QEII Centre, we had 360-degree views of central London, from Big Ben to the London Eye and beyond.
Read more from Steve Brodie, CEO of Electric Cloud @stbrodie
_____________
.IO! .IO! It’s off to a Service Mesh you should go [Gluecon 2017 notes]
http://bit.ly/2rjw4We
Gluecon turned out to be all about a microservice concept called a “service mesh” which was being promoted by Buoyant with Linkerd and IBM/Google/Lyft with Istio. This class of services is a natural evolution of the rush to microservices and something that I’ve written microservice technical architecture on TheNewStack about in the past. READ MORE
_____________
A few things I’ve learned about Kubernetes
https://jvns.ca/blog/2017/06/04/learning-about-kubernetes/
I’ve been learning about Kubernetes at work recently. I only started seriously thinking about it maybe 6 months ago – my partner Kamal has been excited about Kubernetes for a few years (him: “julia! you can run programs without worrying what computers they run on! it is so cool!“, me: “I don’t get it, how is that even possible”), but I understand it a lot better now.
This isn’t a comprehensive explanation or anything, it’s some things I learned along the way that have helped me understand what’s going on.
Read more from Julia Evans @b0rk
_____________
UPCOMING EVENTS
Rob Hirschfeld and Greg Althaus are preparing for a series of upcoming events where they are speaking or just attending. If you are interested in meeting with them at these events please email info@rackn.com.
OTHER NEWSLETTERS
TL;DR: Sometimes paradigm changes demand a rapid response and I believe unifying OpenStack services under Kubernetes has become an such an urgent priority that we must freeze all other work until this effort has been completed.
See Also Rob’s VMblog.com post How is OpenStack so dead AND yet so very alive
That made sense for a multi-vendor project that was deeply integrated with the physical infrastructure and virtualization technologies. The cost of that decision has been high for everyone because we did not converge to shared practices that would drive ease of operations, upgrade or tuning. We ended up with waves of vendors vying to have the the fastest, simplest and openest version.
Tragically, install became an area of competition instead an area of collaboration.
Containers and microservice architecture (as required for Kubernetes and other container schedulers) is providing an opportunity to correct this course. The community is already moving towards containerized services with significant interest in using Kubernetes as the underlay manager for those services. I’ve laid out the arguments for and challenges ahead of this approach in other places.
These technical challenges involve tuning the services for cloud native configuration and immutable designs. They include making sure the project configurations can be injected into containers securely and the infra-service communication can handle container life-cycles. Adjacent concerns like networking and storage also have to be considered. These are all solvable problems that can be more quickly resolved if the community acts together to target just one open underlay.
The critical fact is that the changes are manageable and unifying the solution makes the project stronger.
Using Kubernetes for OpenStack service management does not eliminate or even solve the challenges of deep integration. OpenStack already has abstractions that manage vendor heterogeneity and those abstractions are a key value for the project. Kubernetes solves a different problem: it manages the application services that run OpenStack with a proven, understood pattern. By adopting this pattern fully, we finally give operators consistent, shared and open upgrade, availability and management tooling.
Having a shared, open operational model would help drive OpenStack faster.
There is a risk to this approach: driving Kubernetes as the underlay for OpenStack will force OpenStack services into a more narrow scope as an infrastructure service (aka IaaS). This is a good thing in my opinion. We need multiple abstractions when we build effective IT systems.
The idea that we can build a universal single abstraction for all uses is a dangerous distraction; instead; we need to build platform layers collaborativity.
While initially resisting, I have become enthusiatic about this approach. RackN has been working hard on the upgradable & highly available Kubernetes on Metal prerequisite. We’ve also created prototypes of the fully integrated stack. We believe strongly that this work should be done as a community effort and not within a distro.
My call for a Kubernetes underlay pivot embraces that collaborative approach. If we can keep these platforms focused on their core value then we can build bridges between what we have and our next innovation. What do you think? Is this a good approach? Contact us if you’d like to work together on making this happen.
See Also Rob’s VMblog.com post How is OpenStack so dead AND yet so very alive to SREs?
Today we announced the availability of Digital Rebar Provision, the industry’s first cloud-native physical provisioning utility. We’ve had this in the Digital Rebar community for a few weeks before offering support and response has been great!
By releasing their API-driven provisioning tool as a stand-alone component of the larger Digital Rebar suite, RackN helps DevOps teams break automation bottlenecks in their legacy data centers without disrupting current operations. The stand-alone open utility can be deployed in under 5 minutes and fits into any data center design. RackN also announced a $1,000 starter support and consulting package to further accelerate transition from tools like Cobbler, MaaS or Stacki to the new Golang utility.
“We were seeing SREs suffering from high job turnover,” said Rob Hirschfeld, RackN founder and CEO. “When their integration plans get gridlocked by legacy tooling they quickly either lose patience or political capital. Digital Rebar Provision replaces the legacy tools without process disruption so that everyone can find shared wins early in large SRE initiatives.”
The first cloud-native physical provisioning utility
Data center provisioning is surprisingly complex because it’s caught between cutting edge hardware and arcane protocols and firmware requirements that are difficult to disrupt. The heart of the system is a fickle combination of specific DHCP options, a firmware bootstrap environment (known as PXE), a very lightweight file transfer protocol (TFTP) and operating system specific templating tools like preseed and kickstart. Getting all these pieces to work together with updated APIs without breaking legacy support has been elusive.
By rethinking physical ops in cloud-native terms, RackN has managed to distill out a powerful provisioning tool for DevOps and SRE minded operators who need robust API/CLI, Day 2 Ops, security and control as primary design requirements. By bootstrapping foundational automation with Digital Rebar Provision, DevOps teams lay a foundation for data center operations that improves collaboration between operators and SRE teams: operators enjoy additional control and reuse and SREs get a doorway into building a fully automated process.
A pragmatic path without burning downing the data center
“I’m excited to see RackN providing a pragmatic path from physical boot to provisioning without having to start over and rebuild my data center to get there.” said Dave McCrory, an early cloud and data gravity innovator. “It’s time for the industry to stop splitting physical and cloud IT processes because snowflaked, manual processes slow everyone down. I can’t imagine an easier on-ramp than Digital Rebar Provision”
The RackN Digital Rebar is making it easy for Cobbler, Stacki, MaaS and Forman users to evaluate our RESTful, Golang, Template-based PXE Provisioning utility. Interested users can evaluate the service in minutes on a laptop or engage with RackN for a more comprehensive trail with expert support. The open Provision service works both independently and as part of Digital Rebar’s full life-cycle hybrid control.
S
ee specific features at http://rackn.com/provision/drsa.
Want help starting on this journey? Contact us and we can help.
CaaPuccino: A frothy mix of containers and platforms.
Check out Krish Subramanian’s (@krishnan) Modern Enterprise podcast (audio here) today for a surprisingly deep and thoughtful discussion about how frothy new technologies are impacting Modern Enterprise IT. Of course, we also take some time to throw some fire bombs at the end. You can use my notes below to jump to your favorite topics.
The key takeaways are that portability is hard and we’re still working out the impact of container architecture.
The benefit of the longer interview is that we really dig into the reasons why portability is hard and discuss ways to improve it. My personal SRE posts and those on the RackN blog describe operational processes that improve portability. These are real concerns for all IT organizations because mixed and hybrid models are a fact of life.
If you are not actively making automation that works against multiple infrastructures then you are building technical debt.
Of course, if you just want the snark, then jump forward to 24:00 minutes in where we talk future of Kubernetes, OpenStack and the inverted intersection of the projects.
Krish, thanks for the great discussion!
2:37: Rob intros about Digital Rebar & RackN
4:50: Why our Kubernetes is JUST UPSTREAM
5:35: Where are we going in 5 years > why Rob believes in Hybrid
8:36: Krish believes that Public cloud will be more decentralized
9:25: Docker helps create portability, what else creates portability? Will there be a standard
11:38: Are new standards emerging?
14:50: Why did Platform as a Service not take off?
17:00: What’s the impact on Enterprise? How are developers being impacted?
19:30: Services impact?
22:00: We are having issues with simple, how are we going to scale up?
22:30: Will Kubernetes help us solve governance issues?
24:00: Do you think Kubernetes is going in the right direction?
27:00: How does Kubernetes fit into the Digital Rebar picture?
30:20: Can OpenStack really be managed/run on top of Kubernetes? That seems complex!
35:05 Does OpenStack have a future?
Gene Kim (@RealGeneKim) posted an exclusive Q&A with Rob Hirschfeld (@zehicle) today on IT Technology: Rob Hirschfeld on Containers, Private Clouds, GIFEE, and the Remaining “Underlay Problem.”
Questions from the post:
Read the full Q&A here.
Welcome to the first post of the RackN blog recap of all things SRE. If you have any ideas for this recap or would like to include content please contact us at info@rackn.com.
SRE Items of the Week
Things I Learned Managing Site Reliability for Some of the World’s Busiest Gambling Sites by Ian Miell
INTRO TO POST
For several years I managed the 3rd line site reliability operation for many of the world’s busiest gambling sites, working for a little-known company that built and ran the core backend online software for several businesses that each at peak could take tens of millions of pounds in revenue per hour. I left a couple of years ago, so it’s a good time to reflect on what I learned in the process.
In many ways, what we did was similar to what’s now called an SRE function (I’m going to call us SREs, but the acronym didn’t exist at the time). We were on call, had to respond to incidents, made recommendations for re-engineering, provided robust feedback to developers and customer teams, managed escalations and emergency situations, ran monitoring systems, and so on.
The team I joined was around 5 engineers (all former developers and technical leaders), which grew to around 50 of more mixed experience across multiple locations by the time I left.
I’m going to focus here on process and documentation, since I don’t think they’re talked about usefully enough where I do read about them.
_____
2017 trend lines: When DevOps and hybrid collide by Rob Hirschfeld (@zehicle)
IBM Cloud Computing News
INTRO TO POST
What happens when DevOps methods meet hybrid environments? Following are some emerging trends and my commentary on each.
There are two major casualties as the pace of innovation in IT continues to accelerate: manual processes (non-DevOps) and tightly-coupled software stacks (non-hybrid).
We are changing some things much too quickly for developers and operators to keep up using processes that require human intervention in routine activities like integrated testing or deployment. Furthermore, monolithic platforms—our traditional “duck-and-cover” protection from pace of change—are less attractive for numerous reasons, including slower pace, vendor lock-in and lack of choice.
RECENT SRE AND DEVOPS EVENTS
CloudNativeCon + KubeCon 2017 March 29-30, 2017 in Berlin
IBM Interconnect March in Las Vegas, NV
DevOps Summit
UPCOMING MEETUPS & PODCASTS
Continuous Discussions (#c9d9) Episode 66: Scaling Agile and DevOps in the Enterprise – April 11, 2017 at 10am PT. Rob Hirschfeld a guest in this Electric Cloud podcast.
UPCOMING EVENTS FOR RACKN
Rob Hirschfeld and Greg Althaus are preparing for a series of upcoming events where they are speaking or just attending. If you are interested in meeting with them at these events please email info@rackn.com.
DockerCon 2017 : April 17 – 20, 2017 in Austin, TX
DevOpsDays Austin : May 4-5, 2017 in Austin TX
OpenStack Summit : May 8 – 11, 2017 in Boston, MA
Interop ITX : May 15 – 19, 2017 in Las Vegas, NV
Open Source IT Summit – Tuesday, May 16, 9:00 – 5:00pm : Rob Hirschfeld to speak
Gluecon : May 24 – 25, 2017 in Denver, CO
OTHER NEWSLETTERS
SRE Weekly (@SREWeekly) – Issue #66
Last week, RackN announced our enterprise support for Kubernetes using nothing but upstream Ansible from the project itself. This effort represents years of effort by the RackN founders to keep platforms interoperable via open and shareable operations automation.
That’s why our Digital Rebar approach targets underlay challenges and leverages existing automation tools instead of investing yet another install path.
This week, we added Install Wizard templates to the DC/OS install automation we build in collaboration with Mesosphere last year. That makes it even easier to run DC/OS on physical infrastructure. Like our Kubernetes work, the Digital Rebar automation uses the same community dcos_install.sh that’s used in the community documentation. The difference is that we’re also driving all the underlay prep and configuration automatically.
If this approach appeals to you, contact RackN and join in the open Day 2 revolution.
Interested in seeing the DC/OS install in action? Here’s a demo video:
