In the present study, α-Al2O3 and GDC-0623 xerogel powder converted from sago waste ash were used to prepare mullite. It was found that when the content of α-Al2O3 in the starting composition is increased (and, correspondingly, the total content of silica xerogel is decreased) in the samples sintered at temperatures in the range 1100–1700 °C the amount of the amorphous phase decreases and the crystalline phase of mullite becomes coarse dispersed. Another important observation is that crystalline mullite can be formed at a temperature as low as 1100 °C and a concentration of α-Al2O3 as low as 20 mol%. A complete reaction of mullitization was observed in the sample loaded at 40 mol% α-Al2O3 and sintered at 1600 °C. Therefore, the advantages of using silica xerogel powder as the starting material for mullite preparation are low mullitization temperature, low concentration of α-Al2O3 and unique microstructure. The use of a low-cost raw material (silica xerogel derived from sago waste ash) further reduces the cost of the mullite production and is an appealing alternative to the expensive industrial-grade chemicals.