Journal article: This paper proposes dynamic programming tools for payoffs based on aggregating functions that depend on the current action and the future expected payoff. Some regularity properties are provided on the aggregator to establish existence, uniqueness and computation of the solution to the Bellman equation. Our setting allows to encompass and generalize many previous results based upon additive or non-additive payoff functions.