How can Atlassian better balance evolution of changes in APIs with the pain it causes plugin developers that have to deal with these changes?

[Gary Weaver]

One of the things that is great about Atlassian plugin development has been the community of developers that open-sourced their plugins and other extensions. These examples are of great value and help many get started on the path of extending Atlassian products.

However, as Atlassian quickly develops their products and APIs, sometimes it is even difficult for this community to keep up with the changes in the various Atlassian-provided APIs.

Atlassian currently has a number of Atlassian-supported plugins, including some that they have developed themselves, and these are helpful. In addition, there is a wealth of information in Atlassian provided wikis that the community can help develop. Javadocs are also provided for the APIs (although, I admit I sometimes work with older Javadocs that I find via Google because I couldn't find the latest version of the Javadocs for some APIs Atlassian provides).

However, even though Javadocs, etc. are provided, examples for usage of the various APIs in practice are incomplete. I wonder how Atlassian could better balance changes in these APIs with the needs of those wishing to extend Atlassian products?

[Gary Weaver · Accepted solution]

I think it would be great if Atlassian were to provide a more full set of actual example uses of their APIs, versioned and released along with each Atlassian product, as reference implementations.

Atlassian uses CrowdService, UserAccessor, GroupManager, user and group paging classes, etc. within Confluence, but It is much easier to maintain a codebase with an implementation of the product than it is to maintain information about these APIs in a wiki, because the wiki documentation may look correct, but then oops- maybe it isn't complete or totally up-to-date. I'm not saying that it has to have anything more than a v2 plugin with a basic UI either, and it may not even be that practical. (I hope that no one's ever accused Spring of providing a PetClinic application that they couldn't use in a real veterinarian's office!) I was able to maintain v2 of the Confluence Space User Management plugin for Confluence fairly easily until Confluence v3.5, at which point I'm not sure where I needed to use UserAccessor and when to use CrowdService. Such a reference implementation would have helped in this case.

Atlassian has some wiki documentation about writing authentication classes. (They are kind of "plugins", but they aren't.) However, when Confluence was upgraded, the wiki documentation stayed mostly the same and I wondered whether it just wasn't maintained, so I attempted to upgrade the Confluence Shibboleth Authenticator to an Atlassian v2 plugin. This didn't work. In a ticket, Matt told me that it can't be a v2 plugin, so I tried a v1 plugin. That didn't work either, and I thought I understood that it could use Spring to get access to the beans. That didn't work either, then I was told it had to be loaded via classloader and to use ContainerManager and that ContainerManager had to be called after the beans were instantiated, so it couldn't be called in the contructor of the authenticator. The sad part was that this was the way the plugin was written in the beginning. I had assumed that when I was adding usage of CrowdService that getting bean instances when required was wrong and that I should make it an Atlassian v2 plugin and use constructor injection, but that had been incorrect. It is fine to have those sort of learning experiences, and by a short paragraph or two in the wiki, maybe this would have been avoided. But, having a reference authenticator versioned and released with Confluence 3.5 that uses Atlassian beans such as CrowdService and perhaps had a few choice comments as to why ContainerManager was having to be called at time of need vs. contructor injection as a v2 plugin would have been extremely helpful and would have saved a lot of time.

Another example is that one change in Confluence 2.6 included a breaking API change that required reflection to be used to call the appropriate method. This was fixed in a later version of 2.6, but I had to add the reflection hack that David Peterson came up with to the Confluence Space User Management plugin and it was part of CSUM's codebase until recently.

If you are looking for another major player in the Java development world that provides similar examples, look no further than SpringSource. They've provided the PetClinic application longer than I can remember, and provide additional examples as well for Spring and other frameworks. (Even Spring Roo provides a clinic application.) Another project I've recently worked with that provides examples as part of their codebase is Liferay. They provide a set of examples that they version along with the product that provide sample portlets using various frameworks.

By keeping the examples and documentation in the code, and releasing them with the product, Atlassian could also potentially significantly decrease the amount of work maintaining the wiki documentation, at least over the long-term. (There would need to be a short term project to cleanup and remove examples from the wiki.)

You could argue that instead of writing example code that compiled with the version of the product, why not just add more examples of API usage to the Javadocs. Java developers like to sometimes have examples provided in the Javadoc header for each class, and in some cases this might be helpful to add. However, you can relay concepts more easily about how beans should be wired and best practices in plugin development by providing example code that compiles against the version of the product and its included libraries. I think both options have their place.

I know that there are already some plugins provided and maintained by Atlassian that are open source. However, I think that it would help "eating your own dog food" even more by providing a more full set of examples/samples that are built, tested, and released with the corresponding Atlassian products.

This wouldn't make Atlassian less agile, because they'd be spending less time working on bug fixes after a release. This new documentation (the code being the how-to documentation) would be checked at compile time on every integration build, and the wiki documentation can't even do that. It would also reduce the number of support tickets and increase the number of customers that would rave about Atlassian products' ease of extensibility.

[Gary Weaver]

Recently, Rajendra Kadam noted to me that those wishing to have access to Confluence source code can purchase a 10-user license. (According to Atlassian, you have to be the user associated with the license to have access to the source.) This is a great resource for sure. But in addition to this resource, providing more complete examples of API usage would help Atlassian better understand the impact of changes in the API while providing better documentation of the API.

[Gert-Jan van de Streek]

One of the most helpful things would be to provide a platform where feedback on milestone releases would find it's way back into the Atlassian dev team. Documentation helps, but finding out what you are breaking in an early stage and listening to how this could be better handled works both ways.

[Gary Weaver]

They look to have done this for some milestone releases and not others (i.e. jira.atlassian.com has Confluence 3.4 milestone releases listed, but not those of 3.5). This is a good idea.

[Damian Nowak]

We at Spartez maintain backward compatibility using reflection - most often are able to support full compatibility with all JIRA 4.x versions with it. We hate maintaining different branches for each JIRA. But sometimes you force us to do that or to drop support for older versions. :(

1. Don't change class/interface names - always consider them constant. Class.forName won't see the name change.
2. Never replace a class with an interface. Give it a different name. Otherwise, we get runtime exceptions because of the changes on the bytecode level - INVOKEVIRTUAL can't call an interface method.
3. Never replace a class with an abstract class. Create a new abstract class and make the old class extend the new. Otherwise I'm stuck in creating a new instance.
4. Don't do magic with inheritance like you did with OpenSymphony and Crowd User compatibility in JIRA 4.3. Thanks to your magic most developers didn't get any compilation errors when they moved from JIRA 4.2 and compiled the project against 4.3. But you eventually removed the magic code in JIRA 4.4, so developers got the compilation errors anyway. Please, don't introduce the source code compatibility magic - if you change user handling, let the developer always see the compilation errors.
5. (I will update the answer if something comes to mind)

Apply these hints not only to the public API, but to all the classes. I know that non-public classes don't provide me with any contract or even they are not intended to use by the plugin. But hey - I have the source code of each JIRA minor version, I see the classes and their contracts. You changed the implementation details of some method in JiraWebActionSupport between bugfix versions - it used to call this.getRemoteUser() somewhere, and started to call JiraAuthenticationContext.getUser() - that's OK, I have no problems with it! Implementation details that are not intended to rely on is my problem. Just don't prevent me from writing backward compatible code.

[Gary Weaver]

Great points! You might want to expand on what you mean by "magic code" #4 (at time of writing 2012-03-05).

In the CSUM plugin for Confluence, I had used reflection at one point for compatibility with Confluence (it was actually something David Peterson provided), but eventually since it was a non-paid plugin, we just started updating releases to work with the latest version, and if a customer wanted a plugin for an older version, they would have to download the older version of the plugin. But for a paid plugin, it does seem like reflection to support past versions is a good idea to keep customers happy and continue to provide new features to them, even if they are on an older version of an Atlassian product.

[James Dumay]

Hi Gary,

Internally at Atlassian I've started pushing the use of a set of Annotations that should make it clearer to plugin developers what APIs they can use and not use and filter Javadocs accordingly. Heres an example of Bamboos 'kitchen sink' API doc that contains every class in the application and the new annotation filtered javadoc (incomplete so far). The filtered Javadoc only contains APIs that have been annotated with @PublicApi and @ExperimentalApi. APIs not intended for use by plugin developers use the @Internal annotation and are not included in API docs.

This still needs to gain traction in Atlassians dev teams but hopefully we can start making future releases APIs clearer and cleaner. I'll try to blog about this when we have finished bike sheding.

Cheers,

James

[Gary Weaver]

Please make sure that the paging utilities that are part of the user API don't become internal, if possible. Without those it would make using the user API fairly unscalable. I would opt for Atlassian to hide as few public methods as possible, because if they are useful, they will surely be found and used anyway.

[Ben Coble]

Hey guys,

I saw this post and had to add in my two cents. I'm working with Jira for the first time for my company to develop a plugin. It was quite the chore to trace through the APIs and figure out which pieces I had to do when. Often there would be multiple root pages for how to develop an atlassian or jira gadget and then each of these root pages would split off to smaller pages that each contributed part of what you needed but unfortunately you still had to reference information from the other root pages to get the whole picture. This was quite frustrating to navigate through and the lack of examples that weren't outdated (ie a certain REST resource in the Jira gadget tutorial) made it even harder to figure out how to make a simple Hello World gadget in the first place. Now that I've moused around through the API's and forums for so long I have a better understanding of it all, but it was quite confusing for a while.

Additionally, there are several bugs that consistently appear, even some that haven't been resolved in over 15 MONTHS (https://studio.atlassian.com/browse/AMPS-101). You only have to read the Atlassian comments at the bottom to see how this was a problem for me. There are still problems that I'm trying to fix that I can't find answers for in your forums or the APIs. I ended up writing a guide for my company on how to make a Hello World gadget for anyone that comes after me that has to build one from scratch, because quite frankly, I feel mine is easier to follow and implement than your tutorials. This shouldn't be so.

You guys have a great set of products here that are very useful to a lot of different companies. However, the lack of consistency in your examples and the tutorials that can be mazes makes it very difficult for new developers trying to implement and expand on your product's abilities. This is just one example but I think that it paints a picture of a problem that is more encompassing than the Jira Gadget Tutorials. I hope you guys can resolve this and get a much user-friendly documentation done.

Respectfully,

Ben

[sladey]

G.J.

In JIRA (as with many other products) we ship milestone builds every 2 weeks. You can download these from the Early Access Program link on www.atlassian.com. They can be used to find any issues with your plugins against yet to be released versions of the product. We also have plugin upgrade notes for each major version of JIRA.

In terms of providing feedback, feel free to leave comments on the plugin upgrade notes page or the release notes for any milestone release. We also create an issue on jira.atlassian.com where we collect feedback on our milestone builds (https://jira.atlassian.com/browse/JRA-24125 for JIRA 4.4).

We are very interested in feedback from plugin devs during the release. This forum is another vehicle for such feedback, many of us are reviewing the questions so we will see it.

Daniel talks about a tool that will indicate to plugin devs which APIs are missing/broken against a given version of a product. This tool is inspired by some of the tools that are available with the Eclipse platform. We hope to integrate these tools into the Atlassian SDK.

Cheers,

paul

[Gary Weaver]

Paul, thanks for the info and for the EAP for Jira. The EAP is also a great thing for Confluence. But, I think in addition to something that tells you that your usage of the API is broken (even though that can be done to some extent by just building against the latest version), there should be a working example for how to use the various APIs in each version. These examples could contain Atlassian-suggested workarounds to handle calling the newer and older versions of the API. That is what plugin developers must sometimes do, and Atlassian has not up to this point felt that same pain.

[dan]

Thanks Gary, great question. We are working hard on improving APIs and their stability with deprecation policies, on improving and consolidating dev documentation, and on a tool that notifies plugin developers in advance of upcoming changes that will affect the APIs their plugins use. More suggestions?