golang开发高性能proxy心得

golang开发高性能proxy心得

[文章作者:磨延城 转载请注明原文出处: https://mo2g.com/view/153/ ]

有一段时间没有更新博客了,主要是当爹了,然后各种没时间,久了就搁置了. 这一篇主要分享自己使用golang在开发一个简单proxy功能的过程中总结的心得.先简单介绍一下proxy的功能.

有一段时间没有更新博客了,主要是当爹了,然后各种没时间,久了就搁置了。

这一篇主要分享自己使用golang在开发一个简单proxy功能的过程中总结的心得。先简单介绍一下proxy的功能。

场景:小M使用浏览器访问www.facebook.com

没有proxy


浏览器直接打开www.facebook.com

有proxy


1)浏览器打开proxy.mo2g.com/www.facebook.com

2)proxy程序访问www.facebook.com

3)proxy把www.facebook.com返回的数据转发给浏览器

以上只是proxy的简单用法,更高级的,就根据自己的需求去定制开发吧。这个proxy Demo主要使用了以下包跟框架

compress/gzip包、io/ioutil包、net/http包、net/url包、gin框架(https://github.com/gin-gonic/gin)

默认使用8080端口,废话不多说了,先上代码。

package main

import (
	"compress/gzip"
	"github.com/gin-gonic/gin"
	"io/ioutil"
	"net/http"
	"net/url"
)

var router = gin.Default()

func main() {
	router.Any("/*path", proxy)
	router.Run()
}

func proxy(c *gin.Context) {
	path := c.Param("path")
	if path == "/favicon.ico" {
		c.Status(http.StatusNotFound)
		return
	}
	var (
		req    *http.Request
		resp   *http.Response
		err    error
		body   []byte
		remote *url.URL
	)

	path = c.Query("url")
	path = "https://" + path
	// 转发header、body到新地址
	remote, err = url.Parse(path)
	req = &http.Request{
		URL:        remote,
		Method:     c.Request.Method,
		Header:     c.Request.Header,
		Body:       c.Request.Body,
		Proto:      "HTTP/1.1",
		ProtoMajor: 1,
		ProtoMinor: 1,
	}

	client := &http.Client{}
	resp, err = client.Do(req)
	if err != nil {
		c.JSON(http.StatusForbidden, gin.H{
			"errmsg": err,
		})
		return
	}
	defer resp.Body.Close()
	if resp.Header.Get("Content-Encoding") == "gzip" {
		reader, err := gzip.NewReader(resp.Body)
		if err != nil {
			c.JSON(http.StatusForbidden, gin.H{
				"errmsg": err,
			})
		}
		defer reader.Close()
		body, err = ioutil.ReadAll(reader)
		if err != nil {
			c.JSON(http.StatusForbidden, gin.H{
				"errmsg": err,
			})
		}
	} else {
		body, err = ioutil.ReadAll(resp.Body)
	}

	if err != nil {
		c.JSON(http.StatusForbidden, gin.H{
			"errmsg": err,
		})
		return
	}
	c.Data(http.StatusOK, resp.Header.Get("Content-Type"), body)
}

今晚先写到这了,之后再把剩下的补上。Good Night.


2016/12/3更新,关键代码讲解。

router.Any("/*path", proxy)

把所有类型的请求(GET、POST、PUT、PATCH、HEAD、OPTIONS、DELETE、CONNECT、TRACE)都使用proxy函数来处理。

path := c.Param("path")
if path == "/favicon.ico" {
	c.Status(http.StatusNotFound)
	return
}

忽略favicon.ico请求,某些浏览器会自动请求favicon.ico,例如Chrome。

path = c.Query("url")

获取url的值,例如:proxy.mo2g.com/?url=www.baidu.com

remote, err = url.Parse(path)
req = &http.Request{
	URL:        remote,
	Method:     c.Request.Method,
	Header:     c.Request.Header,
	Body:       c.Request.Body,
	Proto:      "HTTP/1.1",
	ProtoMajor: 1,
	ProtoMinor: 1,
}

client := &http.Client{}
resp, err = client.Do(req)

这段为proxy的关键,c.Request包含了所有的请求数据,我们从中提取Method、Header、Body新建一个Request,URL为url.Parse(path)生成的新地址。

client.Do(req)方法发送请求,返回HTTP回复。(如果返回值err为nil,resp.Body总是非nil的,调用者应该在读取完resp.Body后关闭它)

if resp.Header.Get("Content-Encoding") == "gzip" {
	reader, err := gzip.NewReader(resp.Body)
	defer reader.Close()
} else {
}

判断返回的数据是否经过gzip压缩,是否需要解压数据。

body, err = ioutil.ReadAll(resp.Body)

读取数据直到EOF或遇到error,返回读取的数据和遇到的错误。

就先这样吧,有事留言。(逃

    评论:

    1. 跨境电商运营 回复2017年07月11日 23时13分
      磨者
      非常不错!!!!谢谢!!!

    2. 磨者 957 回复2017年07月26日 04时27分
      磨者
      http://www.buy-hk.org/sh01/index.html
      http://www.buy-hk.org/sw03/index.html
      http://www.buy-hk.org/zz06/index.html
      http://www.buy-hk.org/aw02/index.html
      http://www.buy-hk.org/xz06/index.html
      http://www.buy-hk.org/wh04/index.html
      http://www.buy-hk.org/sy01/index.html
      http://www.buy-hk.org/az09/index.html
      http://www.buy-hk.org/hn05/index.html
      http://www.buy-hk.org/kz01/index.html
      http://www.buy-hk.org/sjz0/index.html
      http://www.buy-hk.org/sp08/index.html
      http://www.buy-hk.org/dy05/index.html
      http://www.buy-hk.org/bu07/index.html
      http://www.buy-hk.org/nc09/index.html
      http://www.buy-hk.org/nz08/index.html
      http://www.buy-hk.org/yh06/index.html
      http://www.buy-hk.org/ay01/index.html
      http://www.buy-hk.org/sk04/index.html
      http://www.buy-hk.org/sz05/index.html
      http://www.buy-hk.org/sd09/index.html
      http://www.buy-hk.org/dc01/index.html
      http://www.buy-hk.org/xy08/index.html
      http://www.buy-hk.org/gb05/index.html
      http://www.buy-hk.org/uz02/index.html
      http://www.buy-hk.org/sn06/index.html
      http://www.buy-hk.org/sr08/index.html
      http://www.buy-hk.org/gz03/index.html
      http://www.buy-hk.org/zh07/index.html
      http://www.buy-hk.org/st01/index.html
      http://www.buy-hk.org/sk08/index.html
      http://www.buy-hk.org/cn03/index.html
      http://www.buy-hk.org/mx06/index.html
      http://www.buy-hk.org/ja06/index.html
      http://www.buy-hk.org/mn09/index.html
      http://www.buy-hk.org/kc07/index.html
      http://www.buy-hk.org/hz03/index.html
      http://www.buy-hk.org/mz05/index.html
      http://www.buy-hk.org/sw09/index.html
      http://www.buy-hk.org/hys8/index.html
      http://www.buy-hk.org/as05/index.html
      http://www.buy-hk.org/qy05/index.html
      http://www.buy-hk.org/ak03/index.html
      http://www.buy-hk.org/ho02/index.html
      http://www.buy-hk.org/jy07/index.html
      http://www.buy-hk.org/hz06/index.html
      http://www.buy-hk.org/mw09/index.html
      http://www.buy-hk.org/hb02/index.html
      http://www.buy-hk.org/hd06/index.html
      http://www.buy-hk.org/bd05/index.html
      http://www.buy-hk.org/cz00/index.html
      http://www.buy-hk.org/xt06/index.html
      http://www.buy-hk.org/pw03/index.html
      http://www.buy-hk.org/cd06/index.html
      http://www.buy-hk.org/hsw9/index.html
      http://www.buy-hk.org/yns0/index.html
      http://www.buy-hk.org/zj00/index.html
      http://www.buy-hk.org/bk03/index.html
      http://www.buy-hk.org/he01/index.html
      http://www.buy-hk.org/hb07/index.html
      http://www.buy-hk.org/nsz1/index.html
      http://www.buy-hk.org/sd06/index.html
      http://www.buy-hk.org/qds9/index.html
      http://www.buy-hk.org/sxs5/index.html
      http://www.buy-hk.org/dts3/index.html
      http://www.buy-hk.org/sx05/index.html
      http://www.buy-hk.org/axa6/index.html
      http://www.buy-hk.org/anah/index.html
      http://www.buy-hk.org/sc90/index.html
      http://www.buy-hk.org/zca9/index.html
      http://www.buy-hk.org/czs6/index.html
      http://www.buy-hk.org/hnz2/index.html
      http://www.buy-hk.org/hks0/index.html
      http://www.buy-hk.org/gzsp/index.html
      http://www.buy-hk.org/gysz/index.html
      http://www.buy-hk.org/jsx3/index.html
      http://www.buy-hk.org/ns08/index.html
      http://www.buy-hk.org/szw0/index.html
      http://www.buy-hk.org/sydw/index.html
      http://www.buy-hk.org/dspm/index.html
      http://www.buy-hk.org/gxsk/index.html
      http://www.buy-hk.org/nsws/index.html
      http://www.buy-hk.org/xj01/index.html
      http://www.buy-hk.org/nx09/index.html
      http://www.buy-hk.org/yc14/index.html
      http://www.buy-hk.org/zk07/index.html
      http://www.buy-hk.org/qh02/index.html
      http://www.buy-hk.org/suxs/index.html
      http://www.buy-hk.org/nms5/index.html
      http://www.buy-hk.org/as09/index.html
      http://www.buy-hk.org/wq01/index.html
      http://www.buy-hk.org/qqhe/index.html
      http://www.buy-hk.org/wezs/index.html
      http://www.buy-hk.org/hhht/index.html
      http://www.buy-hk.org/eeds/index.html
      http://www.buy-hk.org/ycsx/index.html
      http://www.buy-hk.org/wszk/index.html
      http://www.buy-hk.org/jzam/index.html
      http://www.buy-hk.org/xys8/index.html
      http://www.buy-hk.org/zm02/index.html
      http://www.buy-hk.org/xc05/index.html
      http://www.buy-hk.org/gsgs/index.html
      http://www.buy-hk.org/ke09/index.html
      http://www.buy-hk.org/tskp/index.html
      http://www.buy-hk.org/by04/index.html
      http://www.buy-hk.org/pwsu/index.html
      http://www.buy-hk.org/qy06/index.html
      http://www.buy-hk.org/ns09/index.html
      http://www.buy-hk.org/zxzb/index.html
      http://www.buy-hk.org/jc07/index.html
      http://www.buy-hk.org/ww01/index.html
      http://www.buy-hk.org/zy07/index.html
      http://www.buy-hk.org/jq00/index.html
      http://www.buy-hk.org/gx09/index.html
      http://www.buy-hk.org/hszw/index.html
      http://www.buy-hk.org/xn02/index.html
      http://www.buy-hk.org/hdhd/index.html
      http://www.buy-hk.org/hdx8/index.html
      http://www.buy-hk.org/bb04/index.html
      http://www.buy-hk.org/ysxz/index.html

    3. 我来说两句:

        切换  

      磨途歌检测发现,您当前使用的浏览器版本过低,要想使用画板模式,请先更新浏览器

        切换  

      磨途歌随机验证码