Options on upgrading JIRA and plugins with strict firewall and restricted outside network access?

Philip Haynes November 13, 2012

Hi,

I am operating in an environment with strict firewall and outside network access (read none).

The current JIRA and hence other Atlassian plugin architecture is causing significant dramas in that
essentially they require direct outgoing access to the Internet to upgrade and enhance the products. This conflicts with security protocols in place (i.e. no Internet access from production machines). I have attempted manual updates, but this is both time consuming and highly error prone. We need another solution.

The key issue here is that with Jira et al. requiring directly Internet access it bypasses, virus checking / packet inspection and many other security measures of what is downloaded. Further, plugins and other components come from a range of changing URI's making creating URI whitelists impractical.

I have trying to think of a solution that allows my customer to keep their software up to date whilst
sitting within their security policies.

My thinking is that we need a cache of the various updates / plugins where we can do all the
security stuff - prior's to loading the plugins into the secure applications.

Now it turns out that we already have such a thing - it is our Maven / Nexus repo.
This already connects to Atlassian's maven repo to update code during our plugin development cycles.

My question is, what would it take to make it possible to upgrade the Atlassian products and plugins via our Nexus repo or some other means, rather than having to lift up our firewalls to upgrade Atlassian Products directly (which looks doubtful at this stage).

Help greatly appreciated.

Kind Regards,

Philip Haynes

4 answers

1 accepted

2 votes
Answer accepted
ChrisA
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
November 13, 2012

Hi Philip,

Two options that I can suggest for this setup are:

1. Installing plugins via the command line using the Atlassian Plugin SDK.

Assuming you have already downloaded/generated/acquired the plugins needed to install, you can use the atlas-install-plugin command to directly install plugins rather than going through the mess of clicking through the browser and manually uploading them.

Please be aware that you may need to make some configuration changes to run commands:

Running Behind a Corporate Firewall

To run the SDK commands where your Java programs don't have direct access to the Internet, your atlas commands should be run with the -DallowGoogleTracking=false flag. This disables Google web requests during the build process. Equally where authenticated access is available, it is handy to run a local authenticating proxy server such as cntlm and then set the Java http{s} environment variables to point to the proxy (for example, -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128 -Dhttps.proxyHost=localhost -Dhttps.proxyPort=3128 -Dhttp.nonProxyHosts=localhost|127.0.0.*).

2. One of our devs has created a pom.xml file which would potentially allow you to build a new bundled-plugins zip which will update JIRA's existing bundled-plugins. One thing to note is that this will only be able to handle plugins in your maven respository, and will require your Maven repository to be connected to Atlassian Maven Repositories (either by proxy, directly connecting, etc). Also this method is only suitable for jar file and not obr file plugins.

Below is the modified pom.xml -

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <parent>
        <groupId>com.atlassian.jira</groupId>
        <artifactId>jira-bundled-plugins</artifactId>
        <version>5.1.8</version>
    </parent>
    
    <artifactId>jira-plugin-alternative-updater</artifactId>
    <groupId>com.atlassian.labs.jira</groupId>
    <name>Atlassian JIRA Plugin Alternative Updater</name>
    <version>1.0-SNAPSHOT</version>

    <description>
        Provides an augmented bundled plugins jar that can replace the default JIRA bundled plugins jar.
        Run "mvn package" to generate the bundled plugins zip.
    </description>


    <build>
        <plugins>
            <plugin>
                <!-- com.luuuis:smartass-maven-plugin is used in the normal JIRA build process for this artifact -->
                <groupId>com.luuuis</groupId>
                <artifactId>smartass-maven-plugin</artifactId>
                <version>1.2.1</version>
                <executions>
                    <execution>
                        <id>create-bundled-plugins-zip</id>
                        <phase>package</phase>
                        <goals>
                            <goal>archive-dependencies</goal>
                        </goals>
                        <configuration>
                            <includeScope>runtime</includeScope>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
    
    <dependencies>
        <!-- Add dependencies here to include them in the bundled plugins zip. They should have scope runtime-->
        <!-- Using GreenHopper as an example: -->
        <!--
        <dependency>
            <groupId>com.atlassian.jira.plugins</groupId>
            <artifactId>jira-greenhopper-plugin</artifactId>
            <version>5.10.5.2</version>
            <scope>runtime</scope>
        </dependency>
        -->
    </dependencies>

</project>

To use this, set it up as if it was a regular maven2 project. We use 2.1.0. I expect it to work on later versions. What that means is:

  1. Create a directory named "jira-plugin-alternative-updater"
  2. Put that pom.xml into that directory
  3. Add any dependencies to it that you want as bundled plugins
  4. Set the JIRA version to the one you are using (e.g. 5.1.8, 5.2)
  5. In that directory run "mvn clean package"
    1. This will generate an atlassian-bundled-plugins.zip in the target directory
    2. Verify the zip contains the additional files
    3. Verify no plugins in the original atlassian-bundled-plugins.zip bundled with JIRA are missing from the new atlassian-bundled-plugins.zip in the target directory
  6. Replace the atlassian-bundled-plugins.zip that is in JIRA with the atlassian-bundled-plugins.zip that was generated
  7. Start JIRA

This does not grant any additional ability to upgrade plugins through the UPM.

I hope you find this information helpful.

Cheers,

Chris

Philip Haynes November 13, 2012

Thanks Chris - good to seem my "Corporate Firewall" document fragment being quoted back to me ;-)

So what you are saying is that a) I parse our HTTP traffic logs to find the updated plugins and from there b) download them and build the mvn script; and then c) use the atlas-install command to install bundle when we go into production.

OK then - off I go then to get some budget to build this.

Wondering if anyone else would be intereste in such a creation?

Cheers,

Philip

Bob Swift OSS (Bob Swift Atlassian Apps)
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
November 13, 2012

Just a note that atlas-install only supports jar installs and not obr installs - so some installs may fail on dependency problems. There is a request open for this - AMPS-793

0 votes
Dave C
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
March 17, 2013

I've raised a New Feature request in the Atlassian Marketplace for this. Please vote, watch and add your feedback to it - thanks! :)

https://ecosystem.atlassian.net/browse/AMKT-4506

0 votes
ChrisA
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
November 13, 2012

Lol - I didn't even realize that you made that change!

You mentioned that your Maven/Nexus repo is already connected to Atlassian's to update code. Assuming that this infrastructure is in place, the pom.xml is basically used to re-build the bundled-plugins.zip that JIRA is shipped with as well as being able to include any other plugins (from the Atlassian Maven repo). This basically removes UPM from picture, although you wouldn't be able to update any third party plugins this way. You will need to know the Maven locations for the plugins you would like to include and will need Maven installed.

Cheers,

Chris

Philip Haynes November 13, 2012

Yes we have the maven repo infrastrucure and we build our own plugins. I will test the Atlassian plugin update install procedure.

The issue for us is the third party plugins we use that appear to be downloaded across the web. We either parse our HTTP access logs to download and build or stop using 3rd party plugins as keeping our configuration upto date becomes uneconomic.

This is why I was suggesting some helper markings to make log parsing easier.

0 votes
Andy Brook [Plugin People]
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
November 13, 2012

Im pretty sure it wont work. Plugins aren't stored in a maven repository, they're stored on disparate plugin specific locations.

When 'firewalled off' I worked around the 'direct connectivity to internet' firewall problem by setting up a specific internal proxy that could re-proxy out to the internet but could not address internal hosts, in order to get plugin updates. Failing such a questionable solution, manually downloading JAR's is the 'air gap' approach. A PITA for sure, but thats security for you.

Philip Haynes November 13, 2012

Hi Andy,

Realise plugin's aren't currently stored in a maven repo - I believe work is required to make this so. Our development environment uses CLTM to proxy out and as stated above, this isn't available for prod. As stated in the question, our set of plugins is sufficiently complex that manually downloading JARs is a prohibitively fragile and has caused weeks of lost time not getting it working...

Philip

Andy Brook [Plugin People]
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
November 13, 2012

Forgive me I have no idea what CLTM, and I dont see it referenced above :) You are in a hard place.

There is some low level chatter going on between the UPM and Marketplace, in order to know there is an update available. You would have to proxy/stub the UPM backend in order to make this work, which is the hard part, scheduling a retrieval of your plugin lineup isnt too hard and could be done through the last 'build' number, eg

https://marketplace.atlassian.com/download/plugins/com.javahollic.jira.jemh-ui/version/635

then this could be uploaded into a nexus repo, but again, the storage is the least of the problem. Getting JIRA to talk to another backend faking out Marketplace is quite a challenge. I wonder what Atlassian would suggest here, Im sure they have more than a few customers such as yourself behind a firewall. Its no good having a Marketplace if you can't get to it.

It that, or hire an intern :)

Philip Haynes November 13, 2012

Oops misspelt cntlm - http://cntlm.sourceforge.net/

It is the internet proxy that allows all traffic to be monitored.

Yes - I am aware of all the traffic - I have been monitor the logs to see what gets sent out and having the pleasure of reviewing them with the powers that be. It is how I found out about the naughty google tracking code in the documentation I contributed below. But it is not available to me in prod.

Unfortunately there are more than budgetary issues in getting said intern.

Andy Brook [Plugin People]
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
November 13, 2012

Yea I read the bit about the proxy (saw cntlm but couldnt figure if thats what you meant!), what I driving at was having a proxy _inside_ the network that was configured only to allow traffic out of the network. From what you say, security is likely not look favourably on such a solution. Reading below, sounds like you have some fun ahead. I'm sure others would be interested in your solution so do post back!

Suggest an answer

Log in or Sign up to answer