[BUG] Service Map does not rotate with multiple workers

Describe the bug

The service_map is no longer rotating its windows.

The cause of this issue is that with the introduction of #5794, the @SingleThread annotation is not functioning as expected. Only the last @SingleThread processor instances is called by all threads after this commit.

@SingleThread

Fixes a regression in core where @SingleThread annotated processors a…

The change in #5794 was giving the exact same list of Processor instances to each ProcessWorker. This means that for @SingleThread-annotated processors, only one instance was used. However, the intent of @SingleThread is to have a class instance per thread.

This impacted the service_map because of this code:

data-prepper/data-prepper-plugins/service-map-stateful/src/main/java/org/opensearch/dataprepper/plugins/processor/ServiceMapStatefulProcessor.java

Lines 214 to 217 in 6e51f51

    
           if (isMasterInstance()) { 
        
               processIsolatedServiceMapNodes(serviceDependencyRecords); 
        
               rotateWindows(); 
        
           }

Only one thread should be rotating the windows.

However, when this code is reached to check for the master instance/thread, the value of thisProcessorId was always ${threadCount}-1. It was never 0. For example, with 2 workers, this was always 1 and with 4 workers it was always 3.

data-prepper/data-prepper-plugins/service-map-stateful/src/main/java/org/opensearch/dataprepper/plugins/processor/ServiceMapStatefulProcessor.java

Lines 416 to 418 in 6e51f51

    
           private boolean isMasterInstance() { 
        
               return thisProcessorId == 0; 
        
           }

This is the specific code that resulted in this error.

https://github.com/opensearch-project/data-prepper/pull/5794/files#diff-c53121f9198a1e12abd1fb099ce0e9fe0840f13a3b6044d4f2a15772332cf519R239-R240

Each call to this.processorRegistry.swapProcessors(processors); results in swapping out the processors. So only the final processors list is used.

I fixed this in #5902 by providing the ProcessorsProvider directly to each ProcessWorker.

@SingleThread

Fixes a regression in core where @SingleThread annotated processors a…

@SingleThread

Fixes a regression in core where @SingleThread annotated processors a…

@SingleThread

Fixes a regression in core where @SingleThread annotated processors a…

Navigation Menu

Repository navigation

[BUG] Service Map does not rotate with multiple workers #5901

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	if (isMasterInstance()) {
	processIsolatedServiceMapNodes(serviceDependencyRecords);
	rotateWindows();
	}

	private boolean isMasterInstance() {
	return thisProcessorId == 0;
	}

[BUG] Service Map does not rotate with multiple workers #5901

Description

Activity

dlvenable commented on Jul 25, 2025

dlvenable commented on Jul 25, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions