Buckingham Palace empty as Charles and William avoid stay