Infant formulas offer an alternative to breast milk for both normal birth weight (NBW) and immunocompromised intrauterine growth restricted (IUGR) infants. Although the lipid fraction in formulas is often derived from vegetable oils, it is unclear if this alters immunological outcomes relative to milk fats or whether these effects differ between IUGR and NBW infants. We hypothesized that replacing vegetable oil with bovine milk fat in infant formula would improve immune development in IUGR and NBW neonates. Two-day old piglets were selected (NBW, n = 18, IUGR, n = 18) and each group of animals were fed formula based on either vegetable oil (VEG) or bovine milk fat (MILK). Animals were reared until day 23/24 and systemic immune parameters were evaluated. Milk-fat feeding decreased blood neutrophil counts and improved neutrophil function while transiently reducing leucocytes’ expression of genes related to adaptive and innate immunity as well as energy metabolism, following in vitro stimulation by live Staphylococcus epidermidis (whole blood, 2 h). However, there were only a few interactions between milk-fat type and birthweight status. Thus, piglets fed milk-fat-based formula had improved neutrophil maturation and suppressed pro-inflammatory responses, compared to those fed vegetable-oil-based formula