A Solitary Admin with Cloudy Thinking - A Journey of Hope, Despair, and Triumph - Episode 1

[#1]

Preface

The idea for this series of articles came to me while deep in the throes of making the journey from my warm and friendly Server environment to the wispy world of the Clouds. I am the solitary admin referenced in the title. 

I had originally thought this would be a single, longish article. However, as the journey progresses, material for such an article continues to pile up. Emotions are on a roller coaster. "I laughed, I cried, it was better than Cats", as it were. As such, this tale will come in episodes. It currently feels like 5 episodes with me currently at or around episode 3. I say it feels like 5 episodes because, at present and with how this journey feels like it will go after testing and experimenting, I will have something to say over that many installments. I will not, however, rule out more than 5 being the target should I happen to need one to talk about a Bad Thing That Happened. As well, the more I considered it, the more I thought there just might well be other humans not all that far behind me to whom my little epistles might prove helpful.

So, without further adieu....

The Journey Begins

For many of you on a quest like mine, it very likely began in October 2020 when Atlassian announced the sunset date of its Server Edition products. For myself, however, it was merely the reason to get more serious about an effort first visited in 2018.

2018 - First Look

I work for a "SaaS first" company. With a few exceptions, based on functional requirements, it is. Jira and Confluence were firmly in that "not SaaS" camp way back in 2018 because the Cloud versions just didn't own all the functionality needed. Coincidental to that, by measurement, my containerized implementation running on AWS was more reliable than the Cloud offering in a user impact context. For the most part, all were content. I did hear more than a few cries in the dark about how much nicer the UI on the cloud offering was with this being the only substantive reason anyone wanted to move. However, my remit was to provide a solid and flexible environment in which to work so, with commiserating responses given I did agree it was nicer, we continued to sail along the charted course.

When I first joined this company, the environment I inherited seems to have been operated by administrators who very likely meant well. Good intentions alone, however, did not preclude me from landing in a dumpster fire with extra hobos. In my tenure here I have, at least, managed to get the environment back to mere dumpster fire status, a sort of ground state of being for any long running system that doesn't or can't get quite enough housecleaning done. One of the artifacts of this world is a vast constellation of plugins or apps as they are now called for both Jira and Confluence. This was an initial focus of my assessment. Jira and Confluence do... things... out of the box but, to make them truly functional, one needs to extend them. As such, said extensions needed to be validated; can I do all these things cloudy that I do now?

For the purposes of the first look, we'll first dispense with Confluence. While it may well be naivete on my part, it feels like, while labor intensive, Confluence is relatively straightforward. Unlike Jira, there is a useful admin tool, Macro Usage, I found macros from apps used dozens to hundreds of times in apps that simply were not in Cloud. While there wouldn't be significant functional loss or even content formatting losses, I was not prepared to fix that many pages or try to herd enough cats to get content owners to do so. Confluence was a nope.

Jira was a bit more complex to assess. There isn't a simple, definitive way to find where apps are used in the instance. In 2018, I Did Not Care enough to root around in the database to ferret out usage. My apathy knew no bounds when considering grepping a giant XML file to find app keys. Setting that aside, there were just too many apps not represented in Cloud that I would be giving up too much to be fully functional as was extant at the time. While my environment is relatively small, it is very complex to be able to support not only the technical world but also Legal, Human Resources, Marketing, Creative, Content Marketing, Social Media, Finance, and Payroll. Each of those have their own way of working and it was all supported. While I might have been able to move the rather simpler tech/dev world, it was all the non-tech that was a blocker.

Part of my 2018 assessment was to look at Data Center Edition. While my Server Edition implementation had proven rock solid, my sysadmin paranoiac "two is better than one" mindset also was at play. However, in 2018, there were almost as many apps missing for Data Center as there were for Cloud. For all that there was the soothing, "Server apps will likely work fine but Your Mileage May Vary" statements, I wasn't even going to give them their innings to test as I didn't feel comfortable with them in production. Data Center was also functionally a "nope" in 2018.

The other part of my assessment was cost. Both Cloud and Data Center would cost ~50% more than my current annual maintenance including AWS hosting. While Cloud was an outright "no" based on functionality, Data Center was a definite "maybe". Even were I to accept the risk of going live with Server only apps on Data Center, my environment had been solid enough that I couldn't justify the additional cost to myself much less presenting it to the Executive.

2018 drew to a close with yours truly snug and warm in my Server Edition with no plans to look further.

2020 - Looking Again Because Reasons

As with so many of us, 2020 was a year we did stuff in. It progressed with little worth commenting on until we got to the fateful day that Atlassian announced that Server Edition would be sunsetted. Minds were lost. Angst abounded. For me, however, it was impetus to revisit what I had 2 years before.

Some of the angst is not unfounded. For many the 50% increase in annual operating cost is a bitter pill to swallow if, indeed, it could be swallowed at all. A significant number could not even consider Cloud for a variety of either corporate security or statutory reasons. During my assessment, however, the broad statements of "simply not functional" seen many places appear to be untrue in most cases.

I am probably not typical in my mindset when presented with a hugely changed operating paradigm change such as Cloud vs. Hosted. In my mumble years I have administered a  DEC PDP11-780 and a VAX7800. I had super super privileges on a set of Tandem Himalayas. I've administered a wide variety of UNIX servers from a time when there were way too many flavors of UNIX. I've had a variety of UNIX workstations as a daily driver. There was a time where my daily driver workstation was a DEC VT420 terminal with dual RS232 allowing two sessions at a time. I lived on a Mac for many years. I currently have a Windows, Ubuntu Budgie, and Chromebox workstation on my desk that I float between as function, whim, and mood takes me during any given week. I routinely script in several languages including SQL. I have long maintained that a thing made to do a thing has a way to make that thing happen. The trick is finding the thing to make said thing go.

In 2018, there was enough missing that one could not say that Cloud did or could be made to do some things. As I reassessed in 2020 for both Jira and Confluence, the gap between what they could be made to do and what I needed to do narrowed to the point where the venn diagram of those two things was essentially a circle. To be clear: This is not to say that I could get my needed functionality how I had in the past. This time, I could get my "whats" with new and exciting "hows". When looking at the many angry comments about Cloud functionality, one of the common refrains was "<insert thing> cannot be done on Cloud". What I typically saw missing was the full statement of, "<insert thing> cannot be done on Cloud the way I did it before". For the most part, this seems now to only be a true statement if that extension is added.

As with 2018, Confluence can be dispensed with first. Between then and my second deep assessment, enough missing apps had been added to the Cloud offering that it became a non-issue and, coincidentally, those that had not made it were also lightly enough used on my Server implementation that it would not be an onerous task to track them down and simply do it another way or... just not do it. This isn't to say that a Confluence cloud migration would not be tedious but it would not be a case were stuff wasn't done because some formatting macro didn't work. The number of said cases in the tenant where it made a material difference to the content are small enough that corrections by the content owner when they are discovered would be suitable with only a small concerted effort to correct the glaring defects. 

Once again, we find that Jira is more complex to assess. The same types of assessment issues were still there as in 2018. However, it was different this time. In the intervening years, two things happened; many more of the needed apps had made it to Cloud along with the apps themselves growing in capability (both on Server and Cloud) such that overlap in which of the "whats" needed to be accomplished could be done with several "hows" that now existed on Cloud. While not blind to the fact that, for example, JMWE on Server was markedly different from JMWE on Cloud, between what it DOES do and other elements at my disposal on Cloud, I could implement the functionality necessary using changed methodologies. From a functional standpoint, that was the breakthrough. I could see where I could implement essentially all the function I needed. The things I couldn't do would be looked at harshly to determine whether I really need to do them or not. If I really did need to, maybe a blocker. However, in the final assessment, there wasn't any functionality I could not reproduce in Cloud that was a blocker to moving.

Functionally it seems possible now to go Cloud.

The financial assessment differed little between 2018 and 2020. My Server costs would remain largely the same with some uplift in maintenance prices. The delta between Server and Cloud or Data Center remained ~50%. Curiously, the difference between Cloud and Data Center narrowed to be a wash while in 2018, there was a slight but meaningful advantage to Data Center. The one extension to the financial assessment, given there were 3 years until I couldn't renew Server again, do I ride the Server pony until the end and THEN move or go now. Looking at my rather complex environment, rightly or wrongly, I felt that there was the potential for existing Server only apps to cease to be maintained and/or new ones with new and exciting functionality would not be added. To be fair, why would a developer maintain an app for a dying platform and, if the delta was significant, why provide a Server and Data Center app. Three years is rather a long time to go without these things. So... higher cost now but I had a defensible reason to spend more annually now rather than later.

The final reason to start shooting at Cloud now is a more personal one. As noted previously, I am the sole Atlassian administrator. Additionally, I built and maintain the operating environment from the ground up. While I built out the environment so it is robust and easy to live with, I still have to look after it. With the crisp, stateless, location agnostic, and run time provisioned containerization coupled with automated image builds needing only version values of a variety of image elements provided at build time, an application upgrade takes literal minutes. However, I am still on the hook to do them. My operational workload (doing stuff for my customers) is significant. My maintenance items sometimes grow cobwebs because operational workload. My engineering backlog is off crying in a corner due to loneliness. Going to the cloud means almost all of the maintenance and most of the engineering backlog become SEPs (Someone Else's Problem). A giant win for me.

With functionality seemingly possible and a variety of hard and soft reasons driving me to Cloud now, I announced that it is worth a shot and given the go ahead to start looking at this thing for real. I Let It Be Known that the possibility still existed, given what I knew at the time, that I might run into a definitive blocker. However, my backup was Data Center as all of the meaningful apps I need had made it to DC and changing my licenses to DC was all that was needed to march forward essentially status quo and I would have the fun of building out the resilient environment. I had the plan and the backup plan.

In Our Next Episode

I continue down the path apace but despair rains down from the Clouds.

[Pronto Software]

Hi @Mike Rathwell 

As a solitary Admin your title piqued interest and sure-enough, we’re on parallel tracks, including “mumble years” number of carriages hooked to the ole choo-choo train as it clatters through station after station of evolutionary something-or-other; from pre-history mainframes and minis, WAN-LAN, client-server, DBMS, BI-Cubify, interwebs, digital transform everything. “Next station?!?”

(or do we simply arc to where it all began… a la 2001: A Space Odyssey)

I’ve adopted a long-haul project mind-set (which the upper echelons have accepted). Possibly going to end of 2023, mainly to execute a couple of practice migration runs with significant resources thrown at UAT. The add-on app framework differences between Server and Cloud present a major challenge for the major apps (of course these cover the most content data). Failed migration attempts will be subject to much study, the Go-Live run must be perfect. There’s going to be a lot of manual coal shovelling from the hopper into the locomotive boiler.

Look forward to future instalments, next episode will be avidly consumed with Also sprach Zarathustra as a booming soundtrack.

Cheers,

PeteQ

[Tom Lister]

Hi @Mike Rathwell 

perfect timing! I am on the same journey from Data Centre on AWS including the PDP11/VT420 start point.  Prompted by the amount of effort and cost we spend on AWS including staff.

We’re running the migration tools and assessing the differences and work involved. Looks 80% of our time will be on the 20% of projects with customisation and missing cloud plugins. My love affair with scriptrunner  is going to cost me big time.

[Dave Rosenlund _Trundl_]

Great article, @Mike Rathwell!  I'm looking forward to the next installment.  

-dave 

[Mike Rathwell]

Hi @Pronto Software (PeteQ)

You demonstrated exactly the mindset that there is no one right way to get from metal to air. My world will be a bit of what you say but, with the cruft on top of cruft of my world, There is an element of 80/20 (my mandate; my approach will be more like 90/10 because I am a combo of rigorous and lazy) to get it good enough with the least loss of DATA fidelity and fix the functional borks discovered as we find them once up there but get up there fast. Don't given 'em time to complain.

That said, I think your long haul approach with you being the lone Sisyphus. will be rather less stressful.

I am gratified to see that I am not alone in feelings about Cloud with yours and @Tom Lister also seeming to be in the long running been there, broke that camp on many platforms and environments. 

I rather dislike the "Old guys rule" paraphernalia as I think if you have to say such, you do not, in fact, rule. However, I do see a pattern in mindset and approach to this topic where gray power is rather more receptive. Or maybe resigned. Or both.

[Mike Rathwell]

Hi @Tom Lister 

Good to see another who knows what a VT420 is (which, even with my mechanical keyboard fetish, I consider to have had one of the best keyboards ever).

For me, it's not so much the cost of the environment as the aggregate annual will be significantly higher than my total hosted spend. I spent a LONG time on the build out the AWS environment I have along with the containerization to make it as low operational maintenance needed as possible. But it isn't ZERO maintenance and still has to be looked after by this solitary admin from the bare "metal" up to where the users touch/feel it. 

Like you, it will be that same 80/20 20/80 festival to get stuff up there for me. That said, in some respects my fairly deep assessment (albeit without TRYING things) of Cloud a few years ago may have serendipitously paid off. Since I felt I would be making the move eventually, I've had a few years of organically simplifying things including moving Groovyness from ScriptRunner to snippets living in bits of JMWE and/or Automation. I find I will have rather less of that to rewrite than I might have.

I am also fortunate that my user base likes me in spite of my little ways and already are and will be supportive of my efforts and helpful to know what to fix now and what is an annoyance to fix later as we make this journey.

[Mike Rathwell]

Thanks @Dave Rosenlund _Trundl_ 

I am glad you liked it. To my tiny mind, there has been and will be rather a lot of "click this to do that" articles on this sea to sky journey. My focus is/will be on the softer elements of this festival and the broader elements of the transition. Incidentally, my TPM already has a rather large Structure built out for this thing.

[Pronto Software]

Neglected to mention the long haul will include a parallel archiving exercise so we’re not dragging a heavy load of baggage to cloudland. This involves audit style reporting and stakeholder handholding. Won’t be a totally clean slate, but I’m negotiating hard for less-is-best approach. This represents a separate project in its own right, carefully thought out, which takes time, but with a solid business case I’ll maybe score a dedicated resource to do the leg-work.

As for getting there, I’m still mulling over the approach. Get there fast is good, agree on not giving them time to complain. But I’m weighing up a phased approach with more than one site, possibly having Service Management off to the side of Jira Software. This is for business reasons, we’ve grown Service Management into a PM solution for client engagements. Consultants give the customer teams a portal load of request types that follow a project management methodology. Consultants create requests populated with details for a particular chunk of work. It’s functioning well, especially for managing scope creep and change requests and cost estimate arguments. Client meetings are reviews of Queues of requests. With a Covid surge in new business there’s over 500 active Service Desk projects and god-knows how many customer user accounts. We’ll probably do the same with a couple of Confluence sites, one totally external facing for Marketing types with a pretty-me-up add-on to add Jazz to pages, one internal and hardened where actual work happens. These ideas fall from the firm’s ISO and other compliance statuses, we’re a cloud provider and PCI audits are getting tighter at each six monthly round. The Atlassian instance has been squashed onto a slide under an audit microscope. Splitting it out on separate sites gives Bitbucket and Jira Software a cosy corralled off domain. It looks good and logical drawn on a whiteboard, but adds gloopy layers to the migration project.

Cheers,

PeteQ

[Mike Rathwell]

That sounds a lot like the reasons I have, in one of my roadmap timelines for one of the groups to be migrated (not a non-technical one either) the text "Here be Dragons".