We introduce AnyPortal, a novel zero-shot framework for video background replacement that leverages pre-trained diffusion models. Our framework collaboratively integrates the temporal prior of video diffusion models with the relighting capabilities of image diffusion models in a zero-shot setting. To address the critical challenge of foreground consistency, we propose a Refinement Projection Algorithm, which enables pixel-level detail manipulation to ensure precise foreground preservation. AnyPortal is training-free and overcomes the challenges of achieving foreground consistency and temporally coherent relighting.
"A black woman with curly hair in a red suit and sweater holding a computer is smiling in a park outside. The background is clear."
"A black woman with curly hair in a red suit and sweater holding a computer is smiling in a cyberpunk city street with neon lights. The background is clear and bright."
"A beautiful woman holding a glass is in a park outside with warm atmosphere."
"A beautiful woman holding a glass is in a cyberpunk city street with neon lights, with purple lights on the left side in the city street. The background is clear."
"A man with glasses and a hat is talking near a campfire on a night which shine the man bright. The background is clear."
"A man with glasses and a hat is talking on a beach near a sea with waves. The golden sun shines on the sea and the sky is orange with beautiful clouds. The beach is golden."
"A woman wearing earphone is running in a sunny day."
"A woman wearing earphone is running in a dark night under the huge bright moon."
"A cat is standing in a broad living room on a wooden floor. The sunlight shine from the window and shine the cat's shadow and create a warm scene. The background is clear and bright."
"A handsome boy with curly hair and yellow sweater is on a golden beach and sunlight. The background is clear."
"A man with a nice suit is in a classroom. The background is clear."
"A woman in sweater holding an ipad is studying in a bright classroom with white walls and blackboards. The background is clear."
"A beautiful woman bare feet with long hair wearing a dress walk forward with back to the camera, walking towards a castle along a flagstone path. The background is clear."