论文：Sparse Convolutional Neural Networks
作者：Baoyuan Liu, Min Wang, Hassan Foroosh, Marshall F. Tappen, Marianna Penksy
一作单位：Computational Imaging Lab, Computer Science, University of Central Florida, Orlando, FL, USA
录用情况：CVPR'2015
本文将卷积核两次分解，之后对第二次分解得到的“权重”矩阵进行稀疏化，对于ILSVRC2012数据集上训练的一个5层卷积层的网络，经过微调，在1%的性能损失下，获得了90%的稀疏率；尽管稀疏的算法在高度并行的GPU设备上并不吃香，但是对无GPU设备的加速仍有其意义，因此本文设计了在CPU上高效的稀疏矩阵乘法，获得了实质的推理加速；
论文：LCNN: Lookup-based Convolutional Neural Network
作者：Hessam Bagherinezhad, Mohammad Rastegari, Ali Farhadi
一作单位：University of Washington, XNOR.AI
录用情况：CVPR'2017
本文将卷积运算替换为学习一个字典和一组线性组合的权重，并提出了直接从头开始训练的方式，除了一般的分类实验，本文还证明了这种方法在few shot learning的提升，以及对网络初始迭代时收敛水平的提升等；在AlexNet的架构上的修改，该方法获得了55.1%的ImageNet top1准确率以及3.2倍的加速比；
论文：Efficient and Accurate Approximations of Nonlinear Convolutional Networks
作者：Xiangyu Zhang, Jianhua Zou, Xiang Ming, Kaiming He, Jian Sun
一作单位：Microsoft Research
录用情况：CVPR'2015
作者从卷积核沿着空间维度和通道维度展开后的低秩性入手，将其分解为更少的卷积核以及一系列1x1卷积；接着，作者考虑了卷积核后的非线性层的印象，设计了一个更加复杂的优化目标，并给出了优化算法；作者还考虑了多层误差累积现象，将优化目标改为“反对称”的；对于每一层的秩的选择，作者将其视为一个组合问题，用贪心法求解；在这一系列的策略下，SPPnet在字符分类精度损失0.7%的情况下获得了4.5倍加速比；
论文：Speeding up Convolutional Neural Networks with Low Rank Expansions
作者：Max Jaderberg, Andrea Vedaldi and Andrew Zisserman;
一作单位：Visual Geometry Group Department of Engineering Science University of Oxford
录用情况：BMVC'2014
本文在空间维度上对卷积的空间域进行低秩分解，提出了两种分解的Scheme，直接的和间接的，相比于只在空间域上分解卷积核，作者还利用了通道维度上的冗余性，将多个卷积核视为基的线性组合；对于从传统卷积核到分解后的结构，提出了两种优化方法，基于卷积核的重建和基于数据的重建；该可以用到任何一般的卷积层中去。
论文：Structured Convolutions for Efficient Neural Network Design
作者：Yash Bhalgat, Yizhe Zhang, Jamie Menjay Lin, Fatih Porikli
一作单位：Qualcomm AI Research
录用情况：Neurips'2020
本文提出了一种Structured Convolution，将传统的卷积运算分解为sum pooling操作和卷积核更小的卷积操作，从而能够获得更小、更快的推理模型；为了使得分解带来的损失更小，作者还设计了一种Structural Regularization的正则项添加到训练过程中。该方法不亚于先前的各种张量分解或者通道剪枝方法。笔者认为，本文的分解方法很有意思。
内容主要来自MIT EE364a课程slides和book；穿插着网络资源和少量自己的理解（这些内容不保证准确）
本文是重参数化的作者把这一方法做“透”了的一篇文章，在训练时用多路可以重参数化的结构不同的分支，在推理时合并起来；相比于先前的工作（ACNet和RepVGG），训练时的模型本身就在ImageNet上提点了，结合重参数化关于推理速度的卖点，这波是性能与速度的双赢；本文将主要关注代码实现；
模型压缩，包括低秩分解、剪枝、量化、蒸馏等技术，旨在精度损失很小的前提下，获得参数更小、推理速度更快的模型
With irrigation water also came potential problems, the most obvious being the susceptibility of low-lying farmlands to disastrous flooding from and the longer-term problem of salinization.
译文：有了灌溉水也会带来一些潜在的问题，最明显的就是低洼农田对灾难性洪水的敏感性以及长期的盐碱化。
缩写：irrigation water \(\rightarrow\) low-lying farmlands are susceptible to flooding and sanlinization.
分析：这句话乍一看没有找到主语和谓语，仔细分析唯一可能做谓语的只有came，这里是一个介词短语做主语的用法，相似的句子还有：
此外，作为名词性用法，介词短语还可以做宾语或同位语，参考知乎 - 介词短语就这三类用法；
Coordinated social interactions tend to be weak when a colony is first forming, but true colonies provide extra benefits.
译文：当一个群体刚形成时，协调的社会互动往往很弱，但真正的群体聚居提供了额外的好处。
缩写：Colonies provide benefits.
分析：语法上不难。但是陌生的概念、用词以及逻辑关系让人摸不着头脑。什么是协调的社会互动？为什么要用一个转折强调true？
This behavior is especially valuable when the offsite food supplies are restricted or variable in location, as are swarms of aerial insects harvested by swallows.
译文：当食物供给有限或者位置发生变化时，这种行为尤其有价值，如燕子捕获成群的飞虫。
缩写：When foods are restricted, this behavior is valuable.
分析：as表示正如，引导一个被动的状语从句，正常语序应该是as swarms of aerial insects are harvested by swallows. 但是这里倒装了？这里给出了另一个类似的句子：
On balance, the advantages of colonial nesting clearly outweigh the disadvantages, given the many times at which colonial nesting has evolved independently among different groups of birds.
译文：总得来说，考虑到群体筑巢已经在不同鸟类群体中独立的进化了很多次，群体筑巢的好处还是明显大于坏处。
缩写：many times evolution \(\rightarrow\) colonial nesting adva- \(>\) disadva-
分析：这句话可能是想强调多次的独立进化，证实了上述观点，所以把at many times写成了一个从句。
Still lacking, however, is a general framework for testing different hypotheses for the evolution of coloniality.
译文：然而，对于测验不同群体进化的假说，至今仍然缺乏一个通用的框架
缩写：different hypo- for evolu- of colon- lack a framework
分析：正确理解这句话的关键是要知道，lacking是一个形容词而不是什么动名词，因此这是一个倒装句。
Hairy-footed gerbils live in vegetated islands in a sea of sand in the Namib Desert of southern Afria. Habitat use was determined by tracks in the sand and by how quickly they gave up feeding at stations containning seeds mixed with sand.
译文：毛足的gerbils(沙鼠)生活在非洲南部Namib沙漠一片植被覆盖的沙土上。通过gerbils在沙子上的踪迹和它们在放弃混有种子的沙地的速度确定它们的栖息地。
缩写：H.F. G. live in vegetated islands. Tracks & \(\vec{v}\) of giving up feeding \(\rightarrow\) G.'s habitat.
分析：第二句话理解的困难之处在于，一个被动句，and并列两个谓语动词，但是在一个分句中的tracks让人摸不着头脑，如果是their tracks好像还好理解一些；and后句子的station，总是联想到“车站”，这里只是一个抽象的概念，类似于place；关键还是缺乏逻辑，这句话主要在说，研究栖息地的方法，这样下文才好进一步说明观察到沙鼠对于栖息地是如何选择的。
A 2003 Mars Global Surveyor image shows what mission specialists think may be a delta—a fan-shaped network of channels and sediments where a river once flowed into a larger body of water, in this case a lake filling a crater in the southern highlands.
缩写：A image shows a network of channels and sediments. This image depict a river flowed into a large body of water, which results in a lake filling a crater in the southern highlands. This image is thought as a image of delta.
分析：难点在于理解多个从句分别修饰什么对象。what mission specialists think may be a delta说的是照片的内容，——后面的内容也正是照片的内容。where a river once flowed into a larger body of water说的是这个网状结构形成的位置上过去是什么，in this case a lake filling a crater in the southern highlands说的是河水流向水体后的结果。
A computer-generated view of the Martian north polar region shows the extent of what may have been an ancient ocean covering much of the northern lowlands.
缩写：A computer-generated view shows the extent of something covering much of the northern lowlands.
分析：the extend of which这个定语从句的难点在于，对限定词extent的理解。extent表示某事物的范围，且the extent 本身就是宾语从句的主语，因此需要找到后续的动词covering。
The teachers were taken through a program of talking about teaching events, moving on to reflecting about specific issues in a supported, and later an independent, manner.
缩写：The teachers were taken throught a program. This program included talking about events, moving on to reflecting about issues in a supported manner. Later, teachers will do this independently.
分析：in a manner 本是一个固定搭配，表示以某种方式，文中即从supported到independent的方式。
Further observation revealed the tendency of teachers to evaluate events rather than review the contributory factors in a considered manner by, in effect, standing outside the situation.
缩写：Further observation revealed that teachers tend to evalute events rather than review the factors in a considered manner (standing outside the situation).
We need therefore to know how much the climate can vary of its own accord in order to interpret with confidence the extent to which recent changes are natural as opposed to being the result of human activities.
缩写：We need to know how much the climate can vary. In order to interpret the extent of natural changes in recent changes.
The stone between the arms and the body and between the legs in standing figures or the legs and the seat in seated ones was not normally cut away.
分析：出现了很多and、or这样的连词，首先是两个between的并列关系，然后分别是arms和body的并列关系，以及站着或坐着的legs之前的并列关系；
The shift from silent to sound film at the end of the 1920s marks, so far, the most important transformation in motion picture history. Despite all the highly visible technological developments in theatrical and home delivery of the moving image that have occurred over the decades since then, no single innovation has come close to being regarded as a similar kind of watershed.
缩写：The shift from silent to sound film marks the most important transformation. Despite all the developments of the moving image that have occured over the decades since then, no single innovations has been regarded as a similar kind of watershed.
分析：第一句动词是mark，从so far, the most等词中可以得到关键信息。对于第二个长句子，看到despite xxx since then, no xxx being regarded as xxx，基本上可以猜出这句话在强调，电影从无声到有声的转变，是迄今为止最重要的。
Beyond that, the triumph of recorded sound has overshadowed the rich diversity of technological and aesthetic experiments with the visual image that were going forward simultaneously in the 1920s. New color processes, larger or differently shaped screen sizes, multiple-screen projections, even television, were among the developments invented or tried out during the period, sometimes with startling success. The high costs of converting to sound and the early limitations of sound technology were among the factors that suppressed innovations or retarded advancement in these other areas. The introduction of new screen formats was put off for a quarter century, and color, though utilized over the next two decades for special productions, also did not become a norm until the 1950s.
分析：首局先说明了声音记录与同期其他技术成就的关系：Beyond that, the triumph of recorded sound has overshadowed many exp- that were going forward simultaneously in the 1920s. 接着是一些对于technological and aesthetic experiments的举例子，强调了首句的定语从句的内容，即这些其他技术尽管也被发明了，有的有惊人的成功了，（但是还是被声音记录的成功掩盖了）：xxx were invented during the period. 接下来解释为什么，在逻辑上有点绕弯的是，因为声音成本和技术上的问题，（重点都被放在声音上了），导致其他领域的技术发展受限。The high cost and limiations of tech- were among the factors that suppressed innovations in these other areas. 最后说明造成的结果，其他领域技术的引入被推迟：The intro- of xxx was put off, and xxx also did not become a norm util 1950s.
To be sure, their evaluation of the technical flaws in 1920s sound experiments was not so far off the mark, yet they neglected to take into account important new forces in the motion picture field that, in a sense, would not take no for an answer.
分析：这一句承上启下，前文说一些批评家认为有声电影的技术缺陷仍然存在；后文说new forces是什么。阻碍理解短语的包括：
P1: It should be obvious that cetaceans—whales, porpoises, and dolphins—are mammals. They breathe through lungs, not through gills, and give birth to live young. Their streamlined bodies, the absence of hind legs, and the presence of a fluke1 and blowhole2 cannot disguise their affinities with land dwelling mammals. However, unlike the cases of sea otters and pinnipeds (seals, sea lions, and walruses, whose limbs are functional both on land and at sea), it is not easy to envision what the first whales looked like. Extinct but already fully marine cetaceans are known from the fossil record. How was the gap between a walking mammal and a swimming whale bridged? Missing until recently were fossils clearly intermediate, or transitional, between land mammals and cetaceans.
Q1: In paragraph 1, what does the author say about the presence of a blowhole in cetaceans?
A. It clearly indicates that cetaceans are mammals.
B. It cannot conceal the fact that cetaceans are mammals.
C. It is the main difference between cetaceans and land-dwelling mammals.
D. It cannot yield clues about the origins of cetaceans.
cannot disguise their affinities，不能伪装亲近关系，也就是前述证据不足以证明cetaceans are mammals。如果只注意到了cannot disguise 并理解为这些证据使得cetaceans不能再伪装成mammals，则会得到完全相反的答案。
Q2: Which of the following can be inferred from paragraph 1 about early sea otters?
A. It is not difficult to imagine what they looked like.
B. There were great numbers of them.
C. They lived in the sea only.
D. They did not leave many fossil remains.
注意题目中问的是sea otters，文中提到在unlike sea otters条件下，it is not easy to envision。
论文：Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation
作者： Emily Denton, Wojciech Zaremba, Joan Bruna, Yann LeCun, Rob Fergus
录用情况：Neurips'2014
第一作者单位：Dept. of Computer Science, Courant Institute, New York University
笔者最近在做一些用SVD分解卷积核的小实验，并且也在想该如何设计目标函数，让重参数化这件事变成可学习的？然后我发现早在14年就有人做了相关问题的研究；笔者看来，这篇文章的贡献为：
- 给出了一种评估压缩后卷积核好坏的方式，并且通过在线学习的方式，修改了反向传播的梯度（不确定是不是这样）
- 归纳并尝试了对于高维张量的两种分解方法：SVD分解和外积分解；
- 结合聚类方法，针对浅层卷积和深层卷积的不同性质，设计了单色估计方法和双色估计方法；
- 给出一种微调方式：从最浅层开始，压缩一层，然后微调其上所有层，直至性能恢复，如此循环，看能最多压缩多少层；
作者在实验部分用了比较老的模型，仅4层卷积和3层全连接层，现在来看非常简单，但是很适合做一个demo，然而作者原始的C++代码链接已经失效了
