A future detection of the Stochastic Gravitational Wave Background (SGWB) with GW experiments is expected to open a new window on early universe cosmology and on the astrophysics of compact objects. In this paper we study SGWB anisotropies, that can offer new tools to discriminate between different sources of GWs. In particular, the cosmological SGWB inherits its anisotropies both (i) at its production and (ii) during its propagation through our perturbed universe. Concerning (i), we show that it typically leads to anisotropies with order one dependence on frequency. We then compute the effect of (ii) through a Boltzmann approach, including contributions of both large-scale scalar and tensor linearized perturbations. We also compute for the first time the three-point function of the SGWB energy density, which can allow one to extract information on GW non-Gaussianity with interferometers. Finally, we include non-linear effects associated with long wavelength scalar fluctuations, and compute the squeezed limit of the 3-point function for the SGWB density contrast. Such limit satisfies a consistency relation, conceptually similar to what found in the literature for the case of CMB perturbations.