Ken Wyatt, former Indigenous Australians Minister, also called for a shift away from diversity targets.