The synthesis of structured nanoparticles for use in a variety of applications, particularly pharmaceutical materials, has led to a need for improved models for aggregation of colloidal particles. Typically such aggregates are smaller than the Kolmogorov scale and the aggregation is therefore governed by Brownian motion rather than by fluid shear. In such cases, the calculation of Brownian aggregation rate kernels depends crucially on the particle morphology, which influences both the collision cross-section and the particle diffusivity. In this work, we use a robust method to construct rigid fractal aggregates with any desired fractal dimension and particle mass. These fractal clusters are then imported into molecular dynamics simulations and diffusion studies are carried out in the infinite dilution limit. The resulting diffusivity data are used to deduce the relationship between diffusivity (both translational and rotational), fractal dimension, particle mass, and these expressions are compared with predictions developed by other investigators that are based upon estimation of equivalent hydrodynamic radii.