Can getting all pages of space create memory or transaction issues?

Daniel Sák_jnpi3p

Hi, I have 2 questions concerning given simplified code:

@Override
public T doInTransaction() {
    ...
    List<Page> pages = pageManager.getPages(space, true);
    for (Page page : pages) {
         ...
         //some conditions
         ...
         labelManager.addLabel((Labelable) page, someLabel);
         ...
    }
}  

So basically I want iterate over list of pages and give labels to relevant pages.

My questions are:

1. Question: Can getting all pages of given space cause memory issues? like for example having space with too many or too big pages, or combination of both would cause out of memory problems.

2. Question: Is above code a correct way, if I want to add/remove labels ? Or should the transactions be more grained (for example each call to addLabel would have it's own transaction)  and wrapped in some kind of session?

 

Thanks for answers!

 

Hasnae

I would recommend using 

com.atlassian.confluence.api.service.content.ContentService

instead of

com.atlassian.confluence.pages.PageManager

The ContentService by default returns paginated results therefore avoiding any potential issues with memory in case of large data-sets

So something of the sort would do

import com.atlassian.confluence.api.model.content.Content;
import com.atlassian.confluence.api.model.content.ContentType;
import com.atlassian.confluence.api.model.content.Space;
import com.atlassian.confluence.api.model.pagination.PageRequest;
import com.atlassian.confluence.api.model.pagination.PageResponse;
import com.atlassian.confluence.api.model.pagination.SimplePageRequest;
import com.atlassian.confluence.api.service.content.ContentService;
import com.atlassian.plugin.spring.scanner.annotation.imports.ComponentImport;
import org.springframework.stereotype.Component;

import java.util.function.Consumer;
import java.util.function.Predicate;

@Component
public class SpaceContentProcessor {

private static final int LIMIT = 100;
    private final ContentService contentService;

    public SpaceContentProcessor(@ComponentImport final ContentService contentService) {
this.contentService = contentService;
    }

public void process(
final Space space,
            final Predicate<Content> filter,
            final Consumer<Content> handler) {
        internalProcess(space, filter, handler, new SimplePageRequest(0, LIMIT));
    }

private void internalProcess(
final Space space,
            final Predicate<Content> filter,
            final Consumer<Content> handler,
            final PageRequest request) {

final PageResponse<Content> items = contentService.find()
                .withSpace(space)
                .fetchMany(ContentType.PAGE, request);

        items.getResults()
                .stream()
                .filter(filter)
                .forEach(handler);

        if (items.hasMore()) {
final int start = request.getStart() + request.getLimit();
            internalProcess(space, filter, handler, new SimplePageRequest(start, LIMIT));
        }
    }
}

With this you can actually abstract away the filtering and handling you want to do on the content.

You would typically consume the code as follow :

private final SpaceContentProcessor spaceContentProcessor;
private final ContentLabelService contentLabelService

...

spaceContentProcessor.process(
        space, 
        (item) -> { 
// example conditions
            return item.getTitle().contains("something something");
        }, 
        (item) -> {
// example handling
            contentLabelService.addLabels(item.getId(), singletonList(Label.builder("something").build()));
        });

 

Generally avoid using PageManager and friends, we recommend using the services under the java API

com.atlassian.confluence.api.service

 

Should you find limitations to this java API for your use cases, we would want to hear about it !

I have included this code example in this repository https://bitbucket.org/viqueen/confluence-community/src/96168bba3e73dc1aaf9adbbb2f79dc6f8be3384d/src/main/java/com/atlassian/confluence/community/processors/SpaceContentProcessor.java?at=master&fileviewer=file-view-default

I hope this answer helps