Fully saving Online-Newspaper-Articles in Confluence - what is the best way?

Question

Hello, everyone!

For a voluntary, long-term research project (10 users - server license) I have to find a solution to save articles from online newspapers in Confluence in such a way, that they are retained holistically and permanently for documentation and research purposes.

The Confluence-Function for providing web links, for example, is unfortunately not sufficient because it only saves the initial sequence of newspaper articles in Confluence. In addition, this function relies on the external link to the newspaper article being permanently retained as the original source of information.

In fact, relevant source newspaper articles are often deleted or moved to archives after a few days or weeks.

Accordingly, 1:1 copies stored permanently in Confluence are required.

IMPORTANT
The articles must be stored in such a way that they can be arranged by topic (by tags? by sub/menu-structures?) and chronologically according to the date of publication.

CONCLUSION
As far as the aforementioned objective is concerned, I have not yet found an optimal solution.
If possible, this should only be implemented with on-board resources in Confluence so that dependencies on third-party apps are avoided.

With Confluence-Spaces for documentation and knowledge bases, we have not yet achieved optimal structuring, which of course can also be due to a wrong approach.

So far, until today we also have not achieved sufficiently good results, e.g. in terms of formatting, either by copying and pasting or by printing out newspaper articles as PDFs, converting them to Word documents and importing them into Confluence with the aid of an third-party app.

In the aforementioned context, I / we would be grateful for any helpful hint!

Many thanks in advance!

WW · Answer

If the links are deleted, then they're not permanent.

That having been said...

Depending on how technical you are, you could use the Confluence REST API to create and update pages. It's possible that your sources also have REST APIs, but I don't know how many different sources you have or if this would be practical.

With sources that you have no control over, it's difficult to solution this because they could all behave differently.

Have you ever taken a look at the WayBack Machine? It has nothing to do with Confluence, but it might be a potential solution for what you're trying to do. It does have an API, which would centralize your code to go to one spot at least, if you are technically inclined.

Stephan A_ · Answer

Thank you for your reply!

As far as legal considerations or concerns are the focus > we have booked a "workgroup subscription" with an information service that contains articles from different sources (e.g. German Press Agency DPA).

Contractually, we are assured that we may either read required articles online or export them in various formats and use them permanently within our defined team (e.g. PDF, Postscript, Word, ...). Only redistribution to external third parties is not permitted.

In this respect, we are / I am actually looking for a purely technical solution at this point.

Because of the flood of Press-Articles there are permanent links our booked information-service offers - but - they are always deleted after several weeks/months.

WW · Answer

Depending on the country you're in and/or the country from where the articles originate, this might not be legal due to copyright issues.

I'd look into that first before putting a lot of effort into this.

If you want to get the information from the same source, you could look into if that source has permanent links to their articles. Sometimes these links are available at the time of publication.