takeaways
- this paper constructs a particular “homotopy” through $\nu_t := T_{Q_t, b_t} \pi_t$ for Fokker-Planck $\pi_t$ — the particular ($\pi$-independent) evolution of $(Q_t, b_t)$ is exactly the one that ensures that $\nu_t$ solves Hamilton-Jacobi with simply a tilted initial condition
- at $(Q_0, b_0) = (Q, b)$, we have our desired tilted measure
- at large $t$, we flow toward something that is roughly a tilted Gaussian
- this “flow” is strongly coupled with the original flow of $\pi_t$ and allows use of the original denoising oracle
- in the fine-tuning lens, we start with a pre-trained model that turns Gaussian → data, and we design a model that turns fine-tuned Gaussian → fine-tuned data in a way that uses the same computation as the original process
- in tilted transport, there is a clear (linear) map from pre-trained datapoint to fine-tuned datapoint — what is the right generalization to a simple skewing of $\pi$ that isn’t the pushforward of a simple deterministic map?
- ⭐️ suppose that our target posterior $\nu$ is a.c. to $\pi$ with density $f$. in the example of this paper, $-\log f = x^\top Q x + bx$.
- We can still ask reasonable questions: is there a homotopy $\nu_t = f_t \pi_t$ that solves HJE and induces dynamics on $f_t$ such that $\nu_T$ is strongly log-concave (efficiently sampleable)? what are the analogues of SNR and condition number in terms of what $f$ is doing, and are these the fundamental criteria? if $f$ is (approx to) a simple function (ie reweighting the support), what do these $\nu_t$’s look like in an interpretable way?
- i ask this because to me, fine-tuning is often more about shifting the support/upweighting a region, which is easier to view as a simple density against $\pi$ than as a map from samples of $\pi$. however, the proof of Theorem 5 feels applicable when given a density instead of a map
- since Theorem 5 “goes through” (or at least for a position-dependent linearization of the log density), the question boils town to when $\nu_T$ is strongly log-concave, which we might be able to argue about through LSI-type flow along $f_t$ dynamics?
- are these $\nu_t$’s the Wasserstein gradient flow of some functional, if so what
- should be stable to incorrect $A$ — can first learn it
- there are also good bounds related to (box-counting) dimension of $\pi$ for the sample complexity to recover $A$ exactly from (noiseless) samples of $y = Ax$
keep in mind
- think about big-picture of fine-tuning — we seek “continuous deformations/homotopies/annealing” from simple things/the prior to the target/posterior
- pay particular attention to stability w.r.t. the measurement matrix — how can we learn it, and how does error there propagate?
- answer: depends on stability of the resulting $Q$-ODE, which is v stable (just quadratic)
- seek generalizations — recall camillo’s linear measurements classification (in terms of intrinsic dimension of the measure). is there a way to discuss nonlinear tilted transport through geometric properties of nonlinear observations?
GOAL: Assuming a denoising oracle for $\pi$, can we sample efficiently from $y=Ax+w, \; x \sim \pi$ with $w$ an independent noise?
- denoising oracle: for all $\sigma, y$ we have access to the function $\mathbb{E}[x \mid y]$, where $x \sim \pi$ and $y = x + \sigma w$ for $w$ Gaussian. we know that such an oracle allows efficient sampling of $\pi$
- problem setting: now, we are given a known $A$, and seek to sample $y = Ax+\sigma w$
NOTES:
- we can view Ising as a measure on a hypercube that is a quadratic tilt of uniform on the hypercube, which is an explicitly writable prior (since it’s product)
- if $Q$ is taken a GOE, then we get the Sherrington-Kirkpatrick (SK) model
- method is to tilt Gaussian by $(Q_T, b_T)$ gotten as outputs of this ODE, and then run regular reverse SDE to sample
MY QUESTIONS
- why is $b$ allowed to depend on $y$ for the quadratic tilt?
- im still not super comfortable with moving between SDE, Fokker-Planck for density, and Hamilton-Jacobi for potential