Many complex biological systems such as metabolic networks can be divided into functional and organizational subunits, called modules, which provide the flexibility to assemble novel multi-functional hierarchies by a mix and match of simpler components. Here we show that polysaccharide-degrading microbial communities in the ocean can also assemble in a modular fashion. Using synthetic particles made of a variety of polysaccharides commonly found in the ocean, we showed that the particle colonization dynamics of natural bacterioplankton assemblages can be understood as the aggregation of species modules of two main types: a first module type made of narrow niche-range primary degraders, whose dynamics are controlled by particle polysaccharide composition, and a second module type containing broad niche-range, substrate-independent taxa whose dynamics are controlled by interspecific interactions, in particular cross-feeding via organic acids, amino acids and other metabolic byproducts. As a consequence of this modular logic, communities can be predicted to assemble by a sum of substrate-specific primary degrader modules, one for each complex polysaccharide in the particle, connected to a single broad-niche range consumer module. We validate this model by showing that a linear combination of the communities on single-polysaccharide particles accurately predicts community composition on mixed-polysaccharide particles. Our results suggest thus that the assembly of heterotrophic communities that degrade complex organic materials follow simple design principles that can be exploited to engineer heterotrophic microbiomes.