3D Generative Models

Point-E, Self-Hosted: a 154MB Text-to-3D Model That Runs on the Cheapest Card We Rent

OpenAI open-sourced this point-cloud diffusion model in December 2022 and never touched it again after four commits. Its selling point today isn't fidelity — it's size and speed: 41 seconds on a V100 for a 4,096-point colored cloud, with every checkpoint combined weighing just over 7GB.

Point-E · self-hosted

The paper is "Point-E: A System for Generating 3D Point Clouds from Complex Prompts" by Alex Nichol, Heewoo Jun, Prafulla Dhariwal, Pamela Mishkin and Mark Chen, posted to arXiv on 16 December 2022. Code lives at github.com/openai/point-e under MIT. The design is two-stage: a text-to-image diffusion model paints one synthetic view, then a second diffusion model turns that view into a point cloud. The base model emits 1,024 points with 6 channels each (XYZ plus RGB), and a separate 40M upsampler fills that out to 4,096 points. End to end on a single V100 that's about 1.5 minutes, against roughly 12 V100-hours for DreamFusion at the time. That one-to-two-orders-of-magnitude gap was the entire point of the paper.

Now the honest part. Main is frozen at 20 December 2022 — four commits, total — and 64 open issues sit there largely unanswered. The strongest text-to-3D path in the paper depends on a GLIDE text-to-image model fine-tuned on 3D renders, and OpenAI never released those weights (issue #115 is still open with no reply). What you can actually run from text alone is base40M-textvec, which the README itself describes as limited, recognizing simple categories and colors. On COCO CLIP R-Precision, Point-E 1B scores 41.1% / 46.8% (ViT-B/32 and ViT-L/14 scorers) against DreamFusion's 75.1% / 79.7%. Even the official Hugging Face Space is currently sitting in a runtime error.

So why still run it? Three reasons. It is probably the only open text-to-3D pipeline that fits end to end inside 3GB of VRAM, and people working on point-cloud diffusion, SDF reconstruction and 3D generation benchmarks still cite it as a baseline — P-FID and P-IS were introduced by this very paper. It emits an explicit colored point cloud rather than an implicit field, which makes it convenient as a geometry prior, an initializer, or a data-augmentation source. And diffusers only ever adopted Shap-E, not Point-E, so there is no one-line from_pretrained shortcut: you have to spin up a box and clone the original repo. On NexGPU a Tesla V100 32GB — the exact card the paper benchmarked on — is $0.188/GPU-hr. Pick a PyTorch image from the 2,000+ prebuilt ones, SSH in, and you'll have your first cloud in ten minutes.

01 —

Nine Official Checkpoints, and What Each One Actually Is

File sizes are the real Content-Length served from openaipublic.azureedge.net; every one is an fp32 .pt

Version	Parameters	VRAM	Context	Notes
base40M-textvec	40M (width 512 / 12 layers / 8 heads)	154MB fp32 weights / under 3GB for the full chain	1,024 points x 6 channels	The only checkpoint that consumes a text prompt directly, and the default in the text2pointcloud notebook. CLIP ViT-L/14 hands it a single text vector as conditioning, so it handles "a red motorcycle" and falls apart on anything compositional.
base40M / base40M-imagevec / base40M-uncond	40M each (width 512 / 12 layers / 8 heads)	152-155MB fp32 weights each	1,024 points x 6 channels	The image-conditioned trio. base40M conditions on the 16x16 = 256 grid tokens from CLIP ViT-L/14 and is what image2pointcloud loads by default; imagevec uses only the pooled image vector; uncond is the unconditional baseline used for P-FID comparisons.
base300M	300M (width 1024 / 24 layers / 16 heads)	1.16GB fp32 weights / roughly 4GB for the full chain	1,024 points x 6 channels	The quality-versus-speed middle ground, scoring 40.3% / 45.6% on COCO CLIP R-Precision. On a 24GB card you can push a very large batch, which makes it the best value tier for bulk-generating point cloud datasets.
base1B	Nominally 1B (4.64GB of fp32 weights implies ~1.24B; width 2048 / 24 layers / 32 heads)	4.64GB fp32 weights / 8GB and up for the full chain	1,024 points x 6 channels	The strongest tier in the paper at 41.1% / 46.8%. The single file is 4,977,344,725 bytes and one sampling pass takes 28.67 seconds on a V100. Reproducing the repo's banner images means this checkpoint plus a synthetic view generated elsewhere.
upsample	40M (width 512 / 12 layers / 8 heads)	154MB fp32 weights	n_ctx 3072 + cond_ctx 1024 = 4,096 points	Densifies 1,024 base points to 4,096. It is unguided by default — PointCloudSampler drops guidance_scale to 1.0 for every stage after the first, and the text notebook sets it to 0.0 outright. 12.58 seconds on a V100.
sdf (plus pointnet)	sdf ~9.5M / pointnet ~17M	36MB + 67MB, negligible VRAM	Marching cubes grid of 32 or 128	sdf regresses a signed distance field from the point cloud so you can march cubes and write a PLY; it was trained on 2.4M manifold meshes. pointnet is the feature extractor behind P-FID / P-IS and is only needed if you are running the evaluation scripts.

02 —

Picking a Card: Point-E Is FP32-Bound, Not VRAM-Bound

The repo runs fp32 everywhere with no fp16/bf16 path (issue #103 is still open), so peak FP32 throughput matters more than memory capacity

Bulk text-to-point-cloud with base40M-textvec, or reproducing the paper's latency numbers
Tesla V100 32GB$0.188/GPU-hr
The paper's 28.67s and 12.58s were measured on a V100, so matching the card removes a variable — and it happens to be the cheapest tier we rent, with 32GB that this model will never come close to using.
base1B image-to-point-cloud plus upsampling, with a large batch
RTX 4090 24GB$0.540/GPU-hr
4.64GB of weights plus CLIP ViT-L/14 occupies about a quarter of 24GB; the rest goes to batch. Peak FP32 is roughly 82 TFLOPS, five-ish times the V100 class, and pure-fp32 Karras sampling saturates it directly.
SDF mesh reconstruction only — grid_size=128 marching cubes and Blender rendering
RTX 3090 24GB$0.193/GPU-hr
The sdf checkpoint is 36MB and the 2.09M query points of a 128-cubed grid go through in batches of 4,096, so memory pressure is near zero. What you want here is cheap, with 24GB to hold the render pipeline alongside it.
Using Point-E as a baseline and benchmarking it against TRELLIS and Hunyuan3D 2.1 on one box
RTX A6000 48GB$0.817/GPU-hr
TRELLIS asks for at least 16GB, and Hunyuan3D 2.1 needs 10GB for shape, 21GB for texture, 29GB combined. 48GB holds every generation from 2022 to now without swapping machines and re-downloading weights.

03 —

Four Steps to Point-E on a Rented Card

From booting a PyTorch image to writing your first PLY mesh, with the traps marked

01
Boot and install, dodging the CLIP name collision
Launch a PyTorch prebuilt image from the NexGPU console and get in over SSH or Jupyter. setup.py declares clip @ git+https://github.com/openai/CLIP.git, which is OpenAI's CLIP. If you reflexively run pip install clip you get an unrelated PyPI package of the same name and hit module 'clip' has no attribute 'load' at runtime — that is issue #105, open for years. Your image also needs git present or the git+ dependency cannot resolve at all.
```
git clone https://github.com/openai/point-e.git && cd point-e && pip install -e . && pip install git+https://github.com/openai/CLIP.git
```
02
Pin the weight cache to persistent storage
default_cache_dir() in download.py returns os.path.join(os.path.abspath(os.getcwd()), 'point_e_model_cache') — your working directory, not ~/.cache. Whichever directory you start Python from is where several gigabytes land. base1B alone is 4,977,344,725 bytes, so rebuilding the instance means downloading it again. cd into your mounted volume first and pull everything in one pass; all nine checkpoints plus CLIP ViT-L/14 come to about 8GB.
```
cd /workspace && python -c "import torch; from point_e.models.download import load_checkpoint; [load_checkpoint(n, torch.device('cpu')) for n in ['base40M-textvec','base1B','upsample','sdf']]"
```
03
Two-stage sampling: base for structure, upsample for density
PointCloudSampler chains the models: stage one produces 1,024 points, which are passed to stage two as the low_res conditioning to reach 4,096. Defaults are karras_steps (64, 64), sigma_max (120, 160), s_churn (3, 0), with guidance applied only to the first stage. Text generation passes model_kwargs_key_filter=('texts', ''); image generation switches to ('images', '') with PIL images in model_kwargs. If you need it faster, cut the first stage's karras_steps — quality degrades more slowly than you'd expect.
```
sampler = PointCloudSampler(device=device, models=[base_model, upsampler_model], diffusions=[base_diffusion, upsampler_diffusion], num_points=[1024, 4096 - 1024], aux_channels=['R', 'G', 'B'], guidance_scale=[3.0, 0.0], model_kwargs_key_filter=('texts', ''))
```
04
Turn the cloud into a mesh and write a PLY
The pointcloud2mesh notebook ships with grid_size=32, which gives you a blob with a recognizable silhouette; the repo notes that evaluation used 128. A 128-cubed grid is 2.09M query points fed through the sdf model in batches of 4,096 — light on memory, but you will wait. Output is PLY with color on the vertices; expect to recompute normals and decimate before it is comfortable in Blender or MeshLab.
```
from point_e.util.pc_to_mesh import marching_cubes_mesh
mesh = marching_cubes_mesh(pc=pc, model=sdf_model, batch_size=4096, grid_size=128, progress=True)
with open('mesh.ply', 'wb') as f: mesh.write_ply(f)
```

What a Thousand Point Clouds Actually Costs

Using the paper's own V100 breakdown: base1B sampling is 28.67 seconds and upsampling is 12.58, for 41.25 seconds per cloud (the 46.28-second GLIDE stage is unreachable — those weights were never released). A Tesla V100 32GB on NexGPU is $0.188/GPU-hr, which is $0.0000522 per second, so one 4,096-point colored cloud costs 41.25 x 0.0000522 = about $0.00215 — under a quarter of a cent. A thousand of them: 41,250 seconds = 11.46 hours, and 11.46 x 0.188 = $2.15. Move to an RTX 4090 24GB at $0.540/GPU-hr and peak FP32 goes from the V100's ~15 TFLOPS class to roughly 82; even discounting that conservatively to a 3x speedup you get 3.8 hours, and 3.8 x 0.540 = $2.06. Nearly identical spend, two-thirds less wall clock, and enough headroom to fill 24GB with batch. On storage, the nine checkpoints plus CLIP run about 8GB, which at the $0.414/GB-month median is $3.31/month. Compute billing stops the moment the instance stops; storage keeps accruing until you destroy it, so clean up when you're done. Metered per second, no minimum, no setup fee, no quota request.

04 —

FAQ

How much VRAM does Point-E need? Is an 8GB card enough?

Depends on the tier. The base40M-textvec + upsample text path is two 154MB checkpoints plus roughly 1GB for CLIP ViT-L/14 — under 3GB for the whole chain, so 8GB is generous. base300M's weights are 1.16GB, putting the chain around 4GB. base1B is 4.64GB of fp32 weights, and once CLIP and sampling activations are added an 8GB card runs but cannot batch; for bulk generation go straight to 24GB. On NexGPU a Tesla V100 32GB is $0.188/GPU-hr and an RTX 3090 24GB is $0.193/GPU-hr, so squeezing into 8GB saves you essentially nothing.

Is Point-E obsolete? Should anyone still use it for 3D generation?

As a production asset generator, yes, it has been superseded. OpenAI shipped Shap-E in May 2023, whose abstract states plainly that it converges faster than Point-E and reaches comparable or better sample quality while emitting renderable implicit functions. Since then Microsoft TRELLIS (image-large at 1.2B, MIT, officially needs 16GB+) and Tencent Hunyuan3D 2.1 (3.3B shape, 2B texture; 10GB shape, 21GB texture, 29GB combined) produce meshes with PBR textures directly. Point-E still holds up as a point-cloud diffusion baseline, as the smallest working text-to-3D chain at 3GB, and as the origin of P-FID and P-IS. A single RTX A6000 48GB on NexGPU at $0.817/GPU-hr fits all of these at once — run the comparison yourself and the answer becomes obvious.

Why do I get module 'clip' has no attribute 'load' after importing clip?

Because there is an unrelated package called clip on PyPI, and pip install clip fetches that one. Point-E's setup.py declares clip @ git+https://github.com/openai/CLIP.git and you must install OpenAI's version from GitHub. This is issue #105, sitting alongside 63 other open issues in a repo that hasn't seen a commit since its fourth one on 20 December 2022. Run pip uninstall clip, then pip install git+https://github.com/openai/CLIP.git. Pick one of the 2,000+ prebuilt NexGPU images that ships git and a CUDA build of PyTorch and this is a single command.

Can Point-E output go straight into Blender or Unreal?

Not directly. The raw output is 4,096 RGB points with no faces. The repo includes a 36MB sdf model; marching_cubes_mesh regresses the signed distance field and reconstructs a mesh you can write as PLY. Note the notebook defaults to grid_size=32, which only shows a silhouette — the repo says evaluation used 128. Even at 128-cubed, geometry inferred from 4,096 points comes out rounded, with hard edges and fine detail essentially absent. It is fine for placeholder assets, collision proxies and geometry priors, and wrong for finished assets. If you need textured, engine-ready meshes, rent an RTX A6000 48GB at $0.817/GPU-hr and run Hunyuan3D 2.1 instead.

Why does my text-to-3D output look nothing like the paper's figures?

Because the paper's best pipeline is not something you can run. It first generates a synthetic view with a GLIDE text-to-image model fine-tuned on 3D renders (46.28 seconds of the V100 budget) and feeds that image to base1B. Those fine-tuned GLIDE weights were never published; issue #115 asks for them and remains open with no reply. The released text-only route is base40M-textvec, which the README itself calls limited, recognizing simple categories and colors. The practical fix is to bolt on a modern text-to-image model — SDXL, FLUX — generate the view yourself, then run base1B's image-to-point-cloud path. That needs one card holding both models, which is exactly what an RTX 4090 24GB at $0.540/GPU-hr is for.

Does Point-E support fp16 or quantization? Can I just call it from diffusers?

Neither. The repo is fp32 throughout with no fp16/bf16 path — issue #103, "Support for other dtypes", is still open — and there is no official quantized release, which is why you should shop on peak FP32 throughput rather than Tensor Core specs. diffusers adopted Shap-E (ShapEPipeline / ShapEImg2ImgPipeline, weights at openai/shap-e) and never added a Point-E pipeline, so cloning the original repo is the only route, and the official Hugging Face Space is currently throwing a runtime error. Which leaves spinning up your own box as the only way to try it: NexGPU meters per second with no minimum and no quota request, so ten minutes on a Tesla V100 32GB bills out at three cents.

Start building on NexGPU

Enterprise R&D team or solo developer — either way, your first job can be running in minutes.