HLS streaming issue

Discussion:

HLS streaming issue

BURAK EGERCI

2018-03-07 13:39:24 UTC

Dear All,

We are working on HLS streaming. Once # of concurrent sessions reach around 4K, we are experiencing issues regarding the cache algorithm.

Although Cache.config is set to 10minutes for ts files ( url_regex=(.*)\.(ts)$ ttl-in-cache=10m ), lots of requests will be directed to origin server which previously get http 200 from origin server, ATS continues to direct the request to origin server.

Have anyone experienced such an issue? Or does anyone have an idea how to solve this issue?

Please see origin server log belowâŠ ATS sending lots of requests for the same file at the same time although it gets http 200.

176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 181000 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 78192 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"

Please see ATS log at the same time;

1520425086.244 chi=52.214.59.217 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=39879 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.254 chi=34.245.110.191 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40006 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.257 chi=34.242.200.72 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40043 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.258 chi=34.253.240.24 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=39944 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.259 chi=54.171.6.29 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=39951 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.259 chi=34.245.98.243 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40047 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.411 chi=34.245.117.91 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40020 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.414 chi=34.253.239.40 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40150 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.421 chi=34.243.255.27 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40176 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.427 chi=34.249.94.171 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40083 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.428 chi=52.48.159.168 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40028 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.431 chi=54.246.245.203 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=38984 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.435 chi=34.249.44.167 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40103 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.435 chi=54.171.125.63 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=37816 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.435 chi=54.171.125.63 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40198 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"

Thanks & kind regards,
Burak

[Loading Image...

] <http://turkcell.li/iyaani>

Bu elektronik posta ve onunla iletilen butun dosyalar sadece gondericisi tarafindan almasi amaclanan yetkili gercek ya da tuzel kisinin kullanimi icindir. Eger soz konusu yetkili alici degilseniz bu elektronik postanin icerigini aciklamaniz, kopyalamaniz, yonlendirmeniz ve kullanmaniz kesinlikle yasaktir ve bu elektronik postayi derhal silmeniz gerekmektedir.

TURKCELL bu mesajin icerdigi bilgilerin doÄruluÄu veya eksiksiz oldugu konusunda herhangi bir garanti vermemektedir. Bu nedenle bu bilgilerin ne sekilde olursa olsun iceriginden, iletilmesinden, alinmasindan ve saklanmasindan sorumlu degildir. Bu mesajdaki gorusler yalnizca gonderen kisiye aittir ve TURKCELLin goruslerini yansitmayabilir

Bu e-posta bilinen butun bilgisayar viruslerine karsi taranmistir.

________________________________

This e-mail and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the intended recipient you are hereby notified that any dissemination, forwarding, copying or use of any of the information is strictly prohibited, and the e-mail should immediately be deleted.

TURKCELL makes no warranty as to the accuracy or completeness of any information contained in this message and hereby excludes any liability of any kind for the information contained therein or for the information transmission, reception, storage or use of such in any way whatsoever. The opinions expressed in this message belong to sender alone and may not necessarily reflect the opinions of TURKCELL.

This e-mail has been scanned for all known computer viruses.

Vasanth Mathivanan

2018-03-07 14:09:24 UTC

Permalink

Hi,
I have Lot of Questions In front of You:

1.Are You using M3u8 or Direct ts file ?
2.How Many Clients Accessing ( because One client to hits Directly from origin the Other Clients will be loaded on Cache ?
3.Which Storage Method Using like Ram or Hard Disk ?
4.Which Algorithm Using in hdr_rw_config file?
5.Some times happen multiple Concurrent requests so you do need to add Collapse Forwarding Algorithm
6.What Parameter using in records. Config file ?

Thanks & regards

Vasanth M

Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10

From: BURAK EGERCI<mailto:***@turkcell.com.tr>
Sent: Wednesday, March 7, 2018 7:09 PM
To: ***@trafficserver.apache.org<mailto:***@trafficserver.apache.org>
Subject: HLS streaming issue

Dear All,

We are working on HLS streaming. Once # of concurrent sessions reach around 4K, we are experiencing issues regarding the cache algorithm.

Although Cache.config is set to 10minutes for ts files ( url_regex=(.*)\.(ts)$ ttl-in-cache=10m ), lots of requests will be directed to origin server which previously get http 200 from origin server, ATS continues to direct the request to origin server.

Have anyone experienced such an issue? Or does anyone have an idea how to solve this issue?

Please see origin server log below ATS sending lots of requests for the same file at the same time although it gets http 200.

176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 181000 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 78192 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"

Please see ATS log at the same time;

1520425086.244 chi=52.214.59.217 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=39879 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.254 chi=34.245.110.191 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40006 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.257 chi=34.242.200.72 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40043 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.258 chi=34.253.240.24 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=39944 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.259 chi=54.171.6.29 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=39951 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.259 chi=34.245.98.243 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40047 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.411 chi=34.245.117.91 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40020 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.414 chi=34.253.239.40 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40150 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.421 chi=34.243.255.27 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40176 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.427 chi=34.249.94.171 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40083 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.428 chi=52.48.159.168 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40028 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.431 chi=54.246.245.203 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=38984 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.435 chi=34.249.44.167 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40103 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.435 chi=54.171.125.63 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=37816 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.435 chi=54.171.125.63 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40198 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"

Thanks & kind regards,
Burak

[http://www.turkcell.com.tr/downloads/bireysel/img/Tcelldis.gif] <http://turkcell.li/iyaani>

Bu elektronik posta ve onunla iletilen butun dosyalar sadece gondericisi tarafindan almasi amaclanan yetkili gercek ya da tuzel kisinin kullanimi icindir. Eger soz konusu yetkili alici degilseniz bu elektronik postanin icerigini aciklamaniz, kopyalamaniz, yonlendirmeniz ve kullanmaniz kesinlikle yasaktir ve bu elektronik postayi derhal silmeniz gerekmektedir.

TURKCELL bu mesajin icerdigi bilgilerin doðruluðu veya eksiksiz oldugu konusunda herhangi bir garanti vermemektedir. Bu nedenle bu bilgilerin ne sekilde olursa olsun iceriginden, iletilmesinden, alinmasindan ve saklanmasindan sorumlu degildir. Bu mesajdaki gorusler yalnizca gonderen kisiye aittir ve TURKCELLin goruslerini yansitmayabilir

Bu e-posta bilinen butun bilgisayar viruslerine karsi taranmistir.

________________________________

This e-mail and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the intended recipient you are hereby notified that any dissemination, forwarding, copying or use of any of the information is strictly prohibited, and the e-mail should immediately be deleted.

TURKCELL makes no warranty as to the accuracy or completeness of any information contained in this message and hereby excludes any liability of any kind for the information contained therein or for the information transmission, reception, storage or use of such in any way whatsoever. The opinions expressed in this message belong to sender alone and may not necessarily reflect the opinions of TURKCELL.

This e-mail has been scanned for all known computer viruses.

BURAK EGERCI

2018-03-07 14:58:39 UTC

Permalink

Hi,
Thanks for the quick response.

I m using m3u8 file. But not the main m3u8 as I want to specifiy the HLS profile. Lets assume index.m3u8 includes 01.m3u8 and 02.m3u8, I m requesting 01.m3u8.

Once ~4000 concurrent clients are accessing, the issue can be visible. It is not visible if the accessing client number is less than 4000.

Currently Ram storage is configured.

Thanks for Collapse Forwarding algorithm, I will set this up, that may solve the issue. Will let you know after trial.
Here is the records.config;

CONFIG proxy.config.accept_threads INT 5
CONFIG proxy.config.admin.admin_user STRING admin
CONFIG proxy.config.admin.autoconf_port INT 8083
CONFIG proxy.config.admin.number_config_bak INT 3
CONFIG proxy.config.admin.user_id STRING ats
CONFIG proxy.config.alarm.abs_path STRING NULL
CONFIG proxy.config.alarm.bin STRING example_alarm_bin.sh
CONFIG proxy.config.alarm_email STRING ats
CONFIG proxy.config.allocator.debug_filter INT 0
CONFIG proxy.config.allocator.dontdump_iobuffers INT 0
CONFIG proxy.config.allocator.enable_reclaim INT 0
CONFIG proxy.config.allocator.hugepages INT 0
CONFIG proxy.config.allocator.max_overage INT 3
CONFIG proxy.config.allocator.thread_freelist_size INT 1024
CONFIG proxy.config.body_factory.enable_customizations INT 1
CONFIG proxy.config.body_factory.enable_logging INT 0
CONFIG proxy.config.body_factory.response_suppression_mode INT 0
CONFIG proxy.config.body_factory.template_sets_dir STRING etc/trafficserver/body_factory
CONFIG proxy.config.cache.control.filename STRING cache.config
CONFIG proxy.config.cache.enable_read_while_writer INT 1
CONFIG proxy.config.cache.hostdb.sync_frequency INT 0
CONFIG proxy.config.cache.hosting_filename STRING hosting.config
CONFIG proxy.config.cache.http.compatibility.4-2-0-fixup INT 0
CONFIG proxy.config.cache.limits.http.max_alts INT 5
CONFIG proxy.config.cache.max_doc_size INT 0
CONFIG proxy.config.cache.min_average_object_size INT 131072
CONFIG proxy.config.cache.mutex_retry_delay INT 2
CONFIG proxy.config.cache.permit.pinning INT 0
CONFIG proxy.config.cache.ram_cache.algorithm INT 1
CONFIG proxy.config.cache.ram_cache.compress INT 0
CONFIG proxy.config.cache.ram_cache.size INT 34359738368
CONFIG proxy.config.cache.ram_cache.use_seen_filter INT 1
CONFIG proxy.config.cache.ram_cache_cutoff INT 1073741824
CONFIG proxy.config.cache.target_fragment_size INT 1048576
CONFIG proxy.config.cache.threads_per_disk INT 8
CONFIG proxy.config.cluster.cluster_configuration STRING cluster.config
CONFIG proxy.config.cluster.cluster_port INT 8086
CONFIG proxy.config.cluster.ethernet_interface STRING lo
CONFIG proxy.config.cluster.log_bogus_mc_msgs INT 1
CONFIG proxy.config.cluster.mc_group_addr STRING 224.0.1.37
CONFIG proxy.config.cluster.mc_ttl INT 1
CONFIG proxy.config.cluster.mcport INT 8089
CONFIG proxy.config.cluster.rsport INT 8088
CONFIG proxy.config.config_dir STRING etc/trafficserver
CONFIG proxy.config.core_limit INT -1
CONFIG proxy.config.diags.debug.enabled INT 0
CONFIG proxy.config.diags.debug.tags STRING http.*|dns.*
CONFIG proxy.config.diags.show_location INT 0
CONFIG proxy.config.dns.max_dns_in_flight INT 2048
CONFIG proxy.config.dns.nameservers STRING NULL
CONFIG proxy.config.dns.resolv_conf STRING /etc/resolv.conf
CONFIG proxy.config.dns.round_robin_nameservers INT 0
CONFIG proxy.config.dns.search_default_domains INT 0
CONFIG proxy.config.dns.splitDNS.enabled INT 0
CONFIG proxy.config.dns.url_expansions STRING NULL
CONFIG proxy.config.dns.validate_query_name INT 0
CONFIG proxy.config.dump_mem_info_frequency INT 0
CONFIG proxy.config.env_prep STRING example_prep.sh
Thanks & kind regards,
Burak

From: Vasanth Mathivanan [mailto:***@evolutiondigital.com]
Sent: Wednesday, March 7, 2018 5:09 PM
To: ***@trafficserver.apache.org
Subject: COMMERCIAL:RE: HLS streaming issue

Hi,
I have Lot of Questions In front of You:

1.Are You using M3u8 or Direct ts file ?
2.How Many Clients Accessing ( because One client to hits Directly from origin the Other Clients will be loaded on Cache ?
3.Which Storage Method Using like Ram or Hard Disk ?
4.Which Algorithm Using in hdr_rw_config file?
5.Some times happen multiple Concurrent requests so you do need to add Collapse Forwarding Algorithm
6.What Parameter using in records. Config file ?

Thanks & regards

Vasanth M

Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10

From: BURAK EGERCI<mailto:***@turkcell.com.tr>
Sent: Wednesday, March 7, 2018 7:09 PM
To: ***@trafficserver.apache.org<mailto:***@trafficserver.apache.org>
Subject: HLS streaming issue

Dear All,

We are working on HLS streaming. Once # of concurrent sessions reach around 4K, we are experiencing issues regarding the cache algorithm.

Although Cache.config is set to 10minutes for ts files ( url_regex=(.*)\.(ts)$ ttl-in-cache=10m ), lots of requests will be directed to origin server which previously get http 200 from origin server, ATS continues to direct the request to origin server.

Have anyone experienced such an issue? Or does anyone have an idea how to solve this issue?

Please see origin server log below... ATS sending lots of requests for the same file at the same time although it gets http 200.

176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 181000 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 78192 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"

Please see ATS log at the same time;

1520425086.244 chi=52.214.59.217 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=39879 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.254 chi=34.245.110.191 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40006 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.257 chi=34.242.200.72 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40043 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.258 chi=34.253.240.24 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=39944 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.259 chi=54.171.6.29 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=39951 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.259 chi=34.245.98.243 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40047 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.411 chi=34.245.117.91 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40020 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.414 chi=34.253.239.40 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40150 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.421 chi=34.243.255.27 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40176 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.427 chi=34.249.94.171 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40083 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.428 chi=52.48.159.168 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40028 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.431 chi=54.246.245.203 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=38984 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.435 chi=34.249.44.167 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40103 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.435 chi=54.171.125.63 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=37816 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.435 chi=54.171.125.63 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40198 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"

Thanks & kind regards,
Burak

[http://www.turkcell.com.tr/downloads/bireysel/img/Tcelldis.gif]<http://turkcell.li/iyaani>

Bu elektronik posta ve onunla iletilen butun dosyalar sadece gondericisi tarafindan almasi amaclanan yetkili gercek ya da tuzel kisinin kullanimi icindir. Eger soz konusu yetkili alici degilseniz bu elektronik postanin icerigini aciklamaniz, kopyalamaniz, yonlendirmeniz ve kullanmaniz kesinlikle yasaktir ve bu elektronik postayi derhal silmeniz gerekmektedir.

TURKCELL bu mesajin icerdigi bilgilerin doðruluðu veya eksiksiz oldugu konusunda herhangi bir garanti vermemektedir. Bu nedenle bu bilgilerin ne sekilde olursa olsun iceriginden, iletilmesinden, alinmasindan ve saklanmasindan sorumlu degildir. Bu mesajdaki gorusler yalnizca gonderen kisiye aittir ve TURKCELLin goruslerini yansitmayabilir

Bu e-posta bilinen butun bilgisayar viruslerine karsi taranmistir.

________________________________

This e-mail and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the intended recipient you are hereby notified that any dissemination, forwarding, copying or use of any of the information is strictly prohibited, and the e-mail should immediately be deleted.

TURKCELL makes no warranty as to the accuracy or completeness of any information contained in this message and hereby excludes any liability of any kind for the information contained therein or for the information transmission, reception, storage or use of such in any way whatsoever. The opinions expressed in this message belong to sender alone and may not necessarily reflect the opinions of TURKCELL.

This e-mail has been scanned for all known computer viruses.

Jeremy Payne

2018-03-08 01:51:29 UTC

Permalink

I've been running daily load tests for almost the last 6+ months.
6000 simulated users, which is 12k connections at around 17-18gbps.
Before I started playing with the collapse_forwarding plugin, I would see
requests 'leaking' to the origin, when those requests should have been
served by cache.
Now that I have collapse_forwarding tuned to my testing environment,
combined with the associated
records.config parameters, I no longer see 'leaking' origin requests. This
goes for both
100% cached HLS VOD(v-segs=2.7MB a-segs=66KB) and for simulated HLS live.

Post by BURAK EGERCI
Hi,
Thanks for the quick response.
I m using m3u8 file. But not the main m3u8 as I want to specifiy the HLS
profile. Lets assume index.m3u8 includes 01.m3u8 and 02.m3u8, I m
requesting 01.m3u8.
Once ~4000 concurrent clients are accessing, the issue can be visible. It
is not visible if the accessing client number is less than 4000.
Currently Ram storage is configured.
Thanks for Collapse Forwarding algorithm, I will set this up, that may
solve the issue. Will let you know after trial.
Here is the records.config;
CONFIG proxy.config.accept_threads INT 5
CONFIG proxy.config.admin.admin_user STRING admin
CONFIG proxy.config.admin.autoconf_port INT 8083
CONFIG proxy.config.admin.number_config_bak INT 3
CONFIG proxy.config.admin.user_id STRING ats
CONFIG proxy.config.alarm.abs_path STRING NULL
CONFIG proxy.config.alarm.bin STRING example_alarm_bin.sh
CONFIG proxy.config.alarm_email STRING ats
CONFIG proxy.config.allocator.debug_filter INT 0
CONFIG proxy.config.allocator.dontdump_iobuffers INT 0
CONFIG proxy.config.allocator.enable_reclaim INT 0
CONFIG proxy.config.allocator.hugepages INT 0
CONFIG proxy.config.allocator.max_overage INT 3
CONFIG proxy.config.allocator.thread_freelist_size INT 1024
CONFIG proxy.config.body_factory.enable_customizations INT 1
CONFIG proxy.config.body_factory.enable_logging INT 0
CONFIG proxy.config.body_factory.response_suppression_mode INT 0
CONFIG proxy.config.body_factory.template_sets_dir STRING
etc/trafficserver/body_factory
CONFIG proxy.config.cache.control.filename STRING cache.config
CONFIG proxy.config.cache.enable_read_while_writer INT 1
CONFIG proxy.config.cache.hostdb.sync_frequency INT 0
CONFIG proxy.config.cache.hosting_filename STRING hosting.config
CONFIG proxy.config.cache.http.compatibility.4-2-0-fixup INT 0
CONFIG proxy.config.cache.limits.http.max_alts INT 5
CONFIG proxy.config.cache.max_doc_size INT 0
CONFIG proxy.config.cache.min_average_object_size INT 131072
CONFIG proxy.config.cache.mutex_retry_delay INT 2
CONFIG proxy.config.cache.permit.pinning INT 0
CONFIG proxy.config.cache.ram_cache.algorithm INT 1
CONFIG proxy.config.cache.ram_cache.compress INT 0
CONFIG proxy.config.cache.ram_cache.size INT 34359738368
CONFIG proxy.config.cache.ram_cache.use_seen_filter INT 1
CONFIG proxy.config.cache.ram_cache_cutoff INT 1073741824
CONFIG proxy.config.cache.target_fragment_size INT 1048576
CONFIG proxy.config.cache.threads_per_disk INT 8
CONFIG proxy.config.cluster.cluster_configuration STRING cluster.config
CONFIG proxy.config.cluster.cluster_port INT 8086
CONFIG proxy.config.cluster.ethernet_interface STRING lo
CONFIG proxy.config.cluster.log_bogus_mc_msgs INT 1
CONFIG proxy.config.cluster.mc_group_addr STRING 224.0.1.37
CONFIG proxy.config.cluster.mc_ttl INT 1
CONFIG proxy.config.cluster.mcport INT 8089
CONFIG proxy.config.cluster.rsport INT 8088
CONFIG proxy.config.config_dir STRING etc/trafficserver
CONFIG proxy.config.core_limit INT -1
CONFIG proxy.config.diags.debug.enabled INT 0
CONFIG proxy.config.diags.debug.tags STRING http.*|dns.*
CONFIG proxy.config.diags.show_location INT 0
CONFIG proxy.config.dns.max_dns_in_flight INT 2048
CONFIG proxy.config.dns.nameservers STRING NULL
CONFIG proxy.config.dns.resolv_conf STRING /etc/resolv.conf
CONFIG proxy.config.dns.round_robin_nameservers INT 0
CONFIG proxy.config.dns.search_default_domains INT 0
CONFIG proxy.config.dns.splitDNS.enabled INT 0
CONFIG proxy.config.dns.url_expansions STRING NULL
CONFIG proxy.config.dns.validate_query_name INT 0
CONFIG proxy.config.dump_mem_info_frequency INT 0
CONFIG proxy.config.env_prep STRING example_prep.sh
Thanks & kind regards,
Burak
*Sent:* Wednesday, March 7, 2018 5:09 PM
*Subject:* COMMERCIAL:RE: HLS streaming issue
Hi,
1.Are You using M3u8 or Direct ts file ?
2.How Many Clients Accessing ( because One client to hits Directly from
origin the Other Clients will be loaded on Cache ?
3.Which Storage Method Using like Ram or Hard Disk ?
4.Which Algorithm Using in hdr_rw_config file?
5.Some times happen multiple Concurrent requests so you do need to add
Collapse Forwarding Algorithm
6.What Parameter using in records. Config file ?
Thanks & regards
Vasanth M
Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for
Windows 10
*Sent: *Wednesday, March 7, 2018 7:09 PM
*Subject: *HLS streaming issue
Dear All,
We are working on HLS streaming. Once # of concurrent sessions reach
around 4K, we are experiencing issues regarding the cache algorithm.
Although Cache.config is set to 10minutes for ts files (
url_regex=(.*)\.(ts)$ ttl-in-cache=10m ), lots of requests will be directed
to origin server which previously get http 200 from origin server, ATS
continues to direct the request to origin server.
Have anyone experienced such an issue? Or does anyone have an idea how to solve this issue?
*Please see origin server log belowâŠ* ATS sending lots of requests for
the same file at the same time although it gets http 200.
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET
/Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-"
"User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET
/Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-"
"User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET
/Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 181000 "-"
"User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET
/Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-"
"User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET
/Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-"
"User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET
/Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-"
"User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET
/Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-"
"User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET
/Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-"
"User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET
/Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-"
"User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET
/Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-"
"User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET
/Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-"
"User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET
/Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-"
"User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET
/Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-"
"User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET
/Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-"
"User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET
/Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-"
"User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET
/Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-"
"User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET
/Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-"
"User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET
/Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-"
"User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET
/Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 78192 "-"
"User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET
/Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-"
"User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET
/Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-"
"User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET
/Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-"
"User-Agent"
*Please see ATS log at the same time; *
1520425086.244 chi=52.214.59.217 phn=tvgbzcdnes02 php=80 shn=176.235.11.2
url=http://tvgbztcdnes02.channels.turkcelltvplus.com.
tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1
pssc=200 ttms=39879 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN
crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.254 chi=34.245.110.191 phn=tvgbzcdnes02 php=80 shn=176.235.11.2
url=http://tvgbztcdnes02.channels.turkcelltvplus.com.
tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1
pssc=200 ttms=40006 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN
crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.257 chi=34.242.200.72 phn=tvgbzcdnes02 php=80 shn=176.235.11.2
url=http://tvgbztcdnes02.channels.turkcelltvplus.com.
tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1
pssc=200 ttms=40043 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN
crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.258 chi=34.253.240.24 phn=tvgbzcdnes02 php=80 shn=176.235.11.2
url=http://tvgbztcdnes02.channels.turkcelltvplus.com.
tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1
pssc=200 ttms=39944 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN
crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.259 chi=54.171.6.29 phn=tvgbzcdnes02 php=80 shn=176.235.11.2
url=http://tvgbztcdnes02.channels.turkcelltvplus.com.
tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1
pssc=200 ttms=39951 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN
crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.259 chi=34.245.98.243 phn=tvgbzcdnes02 php=80 shn=176.235.11.2
url=http://tvgbztcdnes02.channels.turkcelltvplus.com.
tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1
pssc=200 ttms=40047 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN
crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.411 chi=34.245.117.91 phn=tvgbzcdnes02 php=80 shn=176.235.11.2
url=http://tvgbztcdnes02.channels.turkcelltvplus.com.
tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1
pssc=200 ttms=40020 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN
crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.414 chi=34.253.239.40 phn=tvgbzcdnes02 php=80 shn=176.235.11.2
url=http://tvgbztcdnes02.channels.turkcelltvplus.com.
tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1
pssc=200 ttms=40150 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN
crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.421 chi=34.243.255.27 phn=tvgbzcdnes02 php=80 shn=176.235.11.2
url=http://tvgbztcdnes02.channels.turkcelltvplus.com.
tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1
pssc=200 ttms=40176 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN
crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.427 chi=34.249.94.171 phn=tvgbzcdnes02 php=80 shn=176.235.11.2
url=http://tvgbztcdnes02.channels.turkcelltvplus.com.
tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1
pssc=200 ttms=40083 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN
crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.428 chi=52.48.159.168 phn=tvgbzcdnes02 php=80 shn=176.235.11.2
url=http://tvgbztcdnes02.channels.turkcelltvplus.com.
tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1
pssc=200 ttms=40028 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN
crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.431 chi=54.246.245.203 phn=tvgbzcdnes02 php=80 shn=176.235.11.2
url=http://tvgbztcdnes02.channels.turkcelltvplus.com.
tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1
pssc=200 ttms=38984 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN
crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.435 chi=34.249.44.167 phn=tvgbzcdnes02 php=80 shn=176.235.11.2
url=http://tvgbztcdnes02.channels.turkcelltvplus.com.
tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1
pssc=200 ttms=40103 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN
crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.435 chi=54.171.125.63 phn=tvgbzcdnes02 php=80 shn=176.235.11.2
url=http://tvgbztcdnes02.channels.turkcelltvplus.com.
tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1
pssc=200 ttms=37816 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN
crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.435 chi=54.171.125.63 phn=tvgbzcdnes02 php=80 shn=176.235.11.2
url=http://tvgbztcdnes02.channels.turkcelltvplus.com.
tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1
pssc=200 ttms=40198 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN
crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
Thanks & kind regards,
Burak
<http://turkcell.li/iyaani>
Bu elektronik posta ve onunla iletilen butun dosyalar sadece gondericisi
tarafindan almasi amaclanan yetkili gercek ya da tuzel kisinin kullanimi
icindir. Eger soz konusu yetkili alici degilseniz bu elektronik postanin
icerigini aciklamaniz, kopyalamaniz, yonlendirmeniz ve kullanmaniz
kesinlikle yasaktir ve bu elektronik postayi derhal silmeniz gerekmektedir.
TURKCELL bu mesajin icerdigi bilgilerin doÄruluÄu veya eksiksiz oldugu
konusunda herhangi bir garanti vermemektedir. Bu nedenle bu bilgilerin ne
sekilde olursa olsun iceriginden, iletilmesinden, alinmasindan ve
saklanmasindan sorumlu degildir. Bu mesajdaki gorusler yalnizca gonderen
kisiye aittir ve TURKCELLin goruslerini yansitmayabilir
Bu e-posta bilinen butun bilgisayar viruslerine karsi taranmistir.
------------------------------
This e-mail and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they are
addressed. If you are not the intended recipient you are hereby notified
that any dissemination, forwarding, copying or use of any of the
information is strictly prohibited, and the e-mail should immediately be
deleted.
TURKCELL makes no warranty as to the accuracy or completeness of any
information contained in this message and hereby excludes any liability of
any kind for the information contained therein or for the information
transmission, reception, storage or use of such in any way whatsoever. The
opinions expressed in this message belong to sender alone and may not
necessarily reflect the opinions of TURKCELL.
This e-mail has been scanned for all known computer viruses.

Vasanth Mathivanan

2018-03-08 06:30:13 UTC

Permalink

Hi,
Hdr_rw_live.config File ? We are Using Our Environment Using Like this ?

This Condition based on Every M3u8 file store 5 secs then automatically call m3u8 and ts file every 150 sec store all ts then remove after 151 sec âŠâŠâŠâŠ.All are based on Our live setup like our setup creating M3u8 file every 4 secs so we entered in 5sec in condition and 60 files stored in 150 secs again delete and make new one âŠ.. can you send HLS fragments or chunk value ?

cond %{READ_RESPONSE_HDR_HOOK} [AND]
cond %{PATH} /\.m3u8/
cond %{STATUS} >199 [AND]
cond %{STATUS} <400
rm-header Pragma
rm-header Expires
set-header Cache-Control "max-age=5, public"
cond %{READ_RESPONSE_HDR_HOOK} [AND]
cond %{PATH} /\.ts/
cond %{STATUS} >199 [AND]
cond %{STATUS} <400
rm-header Pragma
rm-header Expires
set-header Cache-Control "max-age=150, public"

Thanks & Regards
Vasanth M

Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10

From: Jeremy Payne<mailto:***@gmail.com>
Sent: Thursday, March 8, 2018 7:21 AM
To: ***@trafficserver.apache.org<mailto:***@trafficserver.apache.org>
Subject: Re: HLS streaming issue

I've been running daily load tests for almost the last 6+ months.
6000 simulated users, which is 12k connections at around 17-18gbps.
Before I started playing with the collapse_forwarding plugin, I would see
requests 'leaking' to the origin, when those requests should have been served by cache.
Now that I have collapse_forwarding tuned to my testing environment, combined with the associated
records.config parameters, I no longer see 'leaking' origin requests. This goes for both
100% cached HLS VOD(v-segs=2.7MB a-segs=66KB) and for simulated HLS live.

On Wed, Mar 7, 2018 at 8:58 AM, BURAK EGERCI <***@turkcell.com.tr<mailto:***@turkcell.com.tr>> wrote:
Hi,
Thanks for the quick response.

I m using m3u8 file. But not the main m3u8 as I want to specifiy the HLS profile. Lets assume index.m3u8 includes 01.m3u8 and 02.m3u8, I m requesting 01.m3u8.

Once ~4000 concurrent clients are accessing, the issue can be visible. It is not visible if the accessing client number is less than 4000.

Currently Ram storage is configured.

Thanks for Collapse Forwarding algorithm, I will set this up, that may solve the issue. Will let you know after trial.
Here is the records.config;

CONFIG proxy.config.accept_threads INT 5
CONFIG proxy.config.admin.admin_user STRING admin
CONFIG proxy.config.admin.autoconf_port INT 8083
CONFIG proxy.config.admin.number_config_bak INT 3
CONFIG proxy.config.admin.user_id STRING ats
CONFIG proxy.config.alarm.abs_path STRING NULL
CONFIG proxy.config.alarm.bin STRING example_alarm_bin.sh
CONFIG proxy.config.alarm_email STRING ats
CONFIG proxy.config.allocator.debug_filter INT 0
CONFIG proxy.config.allocator.dontdump_iobuffers INT 0
CONFIG proxy.config.allocator.enable_reclaim INT 0
CONFIG proxy.config.allocator.hugepages INT 0
CONFIG proxy.config.allocator.max_overage INT 3
CONFIG proxy.config.allocator.thread_freelist_size INT 1024
CONFIG proxy.config.body_factory.enable_customizations INT 1
CONFIG proxy.config.body_factory.enable_logging INT 0
CONFIG proxy.config.body_factory.response_suppression_mode INT 0
CONFIG proxy.config.body_factory.template_sets_dir STRING etc/trafficserver/body_factory
CONFIG proxy.config.cache.control.filename STRING cache.config
CONFIG proxy.config.cache.enable_read_while_writer INT 1
CONFIG proxy.config.cache.hostdb.sync_frequency INT 0
CONFIG proxy.config.cache.hosting_filename STRING hosting.config
CONFIG proxy.config.cache.http.compatibility.4-2-0-fixup INT 0
CONFIG proxy.config.cache.limits.http.max_alts INT 5
CONFIG proxy.config.cache.max_doc_size INT 0
CONFIG proxy.config.cache.min_average_object_size INT 131072
CONFIG proxy.config.cache.mutex_retry_delay INT 2
CONFIG proxy.config.cache.permit.pinning INT 0
CONFIG proxy.config.cache.ram_cache.algorithm INT 1
CONFIG proxy.config.cache.ram_cache.compress INT 0
CONFIG proxy.config.cache.ram_cache.size INT 34359738368
CONFIG proxy.config.cache.ram_cache.use_seen_filter INT 1
CONFIG proxy.config.cache.ram_cache_cutoff INT 1073741824
CONFIG proxy.config.cache.target_fragment_size INT 1048576
CONFIG proxy.config.cache.threads_per_disk INT 8
CONFIG proxy.config.cluster.cluster_configuration STRING cluster.config
CONFIG proxy.config.cluster.cluster_port INT 8086
CONFIG proxy.config.cluster.ethernet_interface STRING lo
CONFIG proxy.config.cluster.log_bogus_mc_msgs INT 1
CONFIG proxy.config.cluster.mc_group_addr STRING 224.0.1.37
CONFIG proxy.config.cluster.mc_ttl INT 1
CONFIG proxy.config.cluster.mcport INT 8089
CONFIG proxy.config.cluster.rsport INT 8088
CONFIG proxy.config.config_dir STRING etc/trafficserver
CONFIG proxy.config.core_limit INT -1
CONFIG proxy.config.diags.debug.enabled INT 0
CONFIG proxy.config.diags.debug.tags STRING http.*|dns.*
CONFIG proxy.config.diags.show_location INT 0
CONFIG proxy.config.dns.max_dns_in_flight INT 2048
CONFIG proxy.config.dns.nameservers STRING NULL
CONFIG proxy.config.dns.resolv_conf STRING /etc/resolv.conf
CONFIG proxy.config.dns.round_robin_nameservers INT 0
CONFIG proxy.config.dns.search_default_domains INT 0
CONFIG proxy.config.dns.splitDNS.enabled INT 0
CONFIG proxy.config.dns.url_expansions STRING NULL
CONFIG proxy.config.dns.validate_query_name INT 0
CONFIG proxy.config.dump_mem_info_frequency INT 0
CONFIG proxy.config.env_prep STRING example_prep.sh
Thanks & kind regards,
Burak

From: Vasanth Mathivanan [mailto:***@evolutiondigital.com<mailto:***@evolutiondigital.com>]
Sent: Wednesday, March 7, 2018 5:09 PM
To: ***@trafficserver.apache.org<mailto:***@trafficserver.apache.org>
Subject: COMMERCIAL:RE: HLS streaming issue

Hi,
I have Lot of Questions In front of You:

1.Are You using M3u8 or Direct ts file ?
2.How Many Clients Accessing ( because One client to hits Directly from origin the Other Clients will be loaded on Cache ?
3.Which Storage Method Using like Ram or Hard Disk ?
4.Which Algorithm Using in hdr_rw_config file?
5.Some times happen multiple Concurrent requests so you do need to add Collapse Forwarding Algorithm
6.What Parameter using in records. Config file ?

Thanks & regards

Vasanth M

Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10

From: BURAK EGERCI<mailto:***@turkcell.com.tr>
Sent: Wednesday, March 7, 2018 7:09 PM
To: ***@trafficserver.apache.org<mailto:***@trafficserver.apache.org>
Subject: HLS streaming issue

Dear All,

We are working on HLS streaming. Once # of concurrent sessions reach around 4K, we are experiencing issues regarding the cache algorithm.

Although Cache.config is set to 10minutes for ts files ( url_regex=(.*)\.(ts)$ ttl-in-cache=10m ), lots of requests will be directed to origin server which previously get http 200 from origin server, ATS continues to direct the request to origin server.

Have anyone experienced such an issue? Or does anyone have an idea how to solve this issue?

Please see origin server log belowâŠ ATS sending lots of requests for the same file at the same time although it gets http 200.

176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 181000 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 78192 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"

Please see ATS log at the same time;

1520425086.244 chi=52.214.59.217 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=39879 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.254 chi=34.245.110.191 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40006 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.257 chi=34.242.200.72 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40043 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.258 chi=34.253.240.24 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=39944 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.259 chi=54.171.6.29 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=39951 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.259 chi=34.245.98.243 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40047 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.411 chi=34.245.117.91 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40020 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.414 chi=34.253.239.40 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40150 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.421 chi=34.243.255.27 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40176 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.427 chi=34.249.94.171 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40083 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.428 chi=52.48.159.168 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40028 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.431 chi=54.246.245.203 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=38984 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.435 chi=34.249.44.167 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40103 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.435 chi=54.171.125.63 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=37816 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.435 chi=54.171.125.63 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40198 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"

Thanks & kind regards,
Burak

[http://www.turkcell.com.tr/downloads/bireysel/img/Tcelldis.gif]<http://turkcell.li/iyaani>

Bu elektronik posta ve onunla iletilen butun dosyalar sadece gondericisi tarafindan almasi amaclanan yetkili gercek ya da tuzel kisinin kullanimi icindir. Eger soz konusu yetkili alici degilseniz bu elektronik postanin icerigini aciklamaniz, kopyalamaniz, yonlendirmeniz ve kullanmaniz kesinlikle yasaktir ve bu elektronik postayi derhal silmeniz gerekmektedir.

TURKCELL bu mesajin icerdigi bilgilerin doÄruluÄu veya eksiksiz oldugu konusunda herhangi bir garanti vermemektedir. Bu nedenle bu bilgilerin ne sekilde olursa olsun iceriginden, iletilmesinden, alinmasindan ve saklanmasindan sorumlu degildir. Bu mesajdaki gorusler yalnizca gonderen kisiye aittir ve TURKCELLin goruslerini yansitmayabilir

Bu e-posta bilinen butun bilgisayar viruslerine karsi taranmistir.

This e-mail and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the intended recipient you are hereby notified that any dissemination, forwarding, copying or use of any of the information is strictly prohibited, and the e-mail should immediately be deleted.

TURKCELL makes no warranty as to the accuracy or completeness of any information contained in this message and hereby excludes any liability of any kind for the information contained therein or for the information transmission, reception, storage or use of such in any way whatsoever. The opinions expressed in this message belong to sender alone and may not necessarily reflect the opinions of TURKCELL.

This e-mail has been scanned for all known computer viruses.

BURAK EGERCI

2018-03-08 07:56:03 UTC

Permalink

Hi,

I am not aware of Hdr_rw_live.config, I believe this is custom config, please confirm.

Thanks & kind regards,
Burak

From: Vasanth Mathivanan [mailto:***@evolutiondigital.com]
Sent: Thursday, March 8, 2018 9:30 AM
To: ***@trafficserver.apache.org
Subject: COMMERCIAL:RE: HLS streaming issue

Hi,
Hdr_rw_live.config File ? We are Using Our Environment Using Like this ?

This Condition based on Every M3u8 file store 5 secs then automatically call m3u8 and ts file every 150 sec store all ts then remove after 151 sec âŠâŠâŠâŠ.All are based on Our live setup like our setup creating M3u8 file every 4 secs so we entered in 5sec in condition and 60 files stored in 150 secs again delete and make new one âŠ.. can you send HLS fragments or chunk value ?

cond %{READ_RESPONSE_HDR_HOOK} [AND]
cond %{PATH} /\.m3u8/
cond %{STATUS} >199 [AND]
cond %{STATUS} <400
rm-header Pragma
rm-header Expires
set-header Cache-Control "max-age=5, public"
cond %{READ_RESPONSE_HDR_HOOK} [AND]
cond %{PATH} /\.ts/
cond %{STATUS} >199 [AND]
cond %{STATUS} <400
rm-header Pragma
rm-header Expires
set-header Cache-Control "max-age=150, public"

Thanks & Regards
Vasanth M

Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10

From: Jeremy Payne<mailto:***@gmail.com>
Sent: Thursday, March 8, 2018 7:21 AM
To: ***@trafficserver.apache.org<mailto:***@trafficserver.apache.org>
Subject: Re: HLS streaming issue

I've been running daily load tests for almost the last 6+ months.
6000 simulated users, which is 12k connections at around 17-18gbps.
Before I started playing with the collapse_forwarding plugin, I would see
requests 'leaking' to the origin, when those requests should have been served by cache.
Now that I have collapse_forwarding tuned to my testing environment, combined with the associated
records.config parameters, I no longer see 'leaking' origin requests. This goes for both
100% cached HLS VOD(v-segs=2.7MB a-segs=66KB) and for simulated HLS live.

On Wed, Mar 7, 2018 at 8:58 AM, BURAK EGERCI <***@turkcell.com.tr<mailto:***@turkcell.com.tr>> wrote:
Hi,
Thanks for the quick response.

I m using m3u8 file. But not the main m3u8 as I want to specifiy the HLS profile. Lets assume index.m3u8 includes 01.m3u8 and 02.m3u8, I m requesting 01.m3u8.

Once ~4000 concurrent clients are accessing, the issue can be visible. It is not visible if the accessing client number is less than 4000.

Currently Ram storage is configured.

Thanks for Collapse Forwarding algorithm, I will set this up, that may solve the issue. Will let you know after trial.
Here is the records.config;

CONFIG proxy.config.accept_threads INT 5
CONFIG proxy.config.admin.admin_user STRING admin
CONFIG proxy.config.admin.autoconf_port INT 8083
CONFIG proxy.config.admin.number_config_bak INT 3
CONFIG proxy.config.admin.user_id STRING ats
CONFIG proxy.config.alarm.abs_path STRING NULL
CONFIG proxy.config.alarm.bin STRING example_alarm_bin.sh
CONFIG proxy.config.alarm_email STRING ats
CONFIG proxy.config.allocator.debug_filter INT 0
CONFIG proxy.config.allocator.dontdump_iobuffers INT 0
CONFIG proxy.config.allocator.enable_reclaim INT 0
CONFIG proxy.config.allocator.hugepages INT 0
CONFIG proxy.config.allocator.max_overage INT 3
CONFIG proxy.config.allocator.thread_freelist_size INT 1024
CONFIG proxy.config.body_factory.enable_customizations INT 1
CONFIG proxy.config.body_factory.enable_logging INT 0
CONFIG proxy.config.body_factory.response_suppression_mode INT 0
CONFIG proxy.config.body_factory.template_sets_dir STRING etc/trafficserver/body_factory
CONFIG proxy.config.cache.control.filename STRING cache.config
CONFIG proxy.config.cache.enable_read_while_writer INT 1
CONFIG proxy.config.cache.hostdb.sync_frequency INT 0
CONFIG proxy.config.cache.hosting_filename STRING hosting.config
CONFIG proxy.config.cache.http.compatibility.4-2-0-fixup INT 0
CONFIG proxy.config.cache.limits.http.max_alts INT 5
CONFIG proxy.config.cache.max_doc_size INT 0
CONFIG proxy.config.cache.min_average_object_size INT 131072
CONFIG proxy.config.cache.mutex_retry_delay INT 2
CONFIG proxy.config.cache.permit.pinning INT 0
CONFIG proxy.config.cache.ram_cache.algorithm INT 1
CONFIG proxy.config.cache.ram_cache.compress INT 0
CONFIG proxy.config.cache.ram_cache.size INT 34359738368
CONFIG proxy.config.cache.ram_cache.use_seen_filter INT 1
CONFIG proxy.config.cache.ram_cache_cutoff INT 1073741824
CONFIG proxy.config.cache.target_fragment_size INT 1048576
CONFIG proxy.config.cache.threads_per_disk INT 8
CONFIG proxy.config.cluster.cluster_configuration STRING cluster.config
CONFIG proxy.config.cluster.cluster_port INT 8086
CONFIG proxy.config.cluster.ethernet_interface STRING lo
CONFIG proxy.config.cluster.log_bogus_mc_msgs INT 1
CONFIG proxy.config.cluster.mc_group_addr STRING 224.0.1.37
CONFIG proxy.config.cluster.mc_ttl INT 1
CONFIG proxy.config.cluster.mcport INT 8089
CONFIG proxy.config.cluster.rsport INT 8088
CONFIG proxy.config.config_dir STRING etc/trafficserver
CONFIG proxy.config.core_limit INT -1
CONFIG proxy.config.diags.debug.enabled INT 0
CONFIG proxy.config.diags.debug.tags STRING http.*|dns.*
CONFIG proxy.config.diags.show_location INT 0
CONFIG proxy.config.dns.max_dns_in_flight INT 2048
CONFIG proxy.config.dns.nameservers STRING NULL
CONFIG proxy.config.dns.resolv_conf STRING /etc/resolv.conf
CONFIG proxy.config.dns.round_robin_nameservers INT 0
CONFIG proxy.config.dns.search_default_domains INT 0
CONFIG proxy.config.dns.splitDNS.enabled INT 0
CONFIG proxy.config.dns.url_expansions STRING NULL
CONFIG proxy.config.dns.validate_query_name INT 0
CONFIG proxy.config.dump_mem_info_frequency INT 0
CONFIG proxy.config.env_prep STRING example_prep.sh
Thanks & kind regards,
Burak

From: Vasanth Mathivanan [mailto:***@evolutiondigital.com<mailto:***@evolutiondigital.com>]
Sent: Wednesday, March 7, 2018 5:09 PM
To: ***@trafficserver.apache.org<mailto:***@trafficserver.apache.org>
Subject: COMMERCIAL:RE: HLS streaming issue

Hi,
I have Lot of Questions In front of You:

1.Are You using M3u8 or Direct ts file ?
2.How Many Clients Accessing ( because One client to hits Directly from origin the Other Clients will be loaded on Cache ?
3.Which Storage Method Using like Ram or Hard Disk ?
4.Which Algorithm Using in hdr_rw_config file?
5.Some times happen multiple Concurrent requests so you do need to add Collapse Forwarding Algorithm
6.What Parameter using in records. Config file ?

Thanks & regards

Vasanth M

Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10

From: BURAK EGERCI<mailto:***@turkcell.com.tr>
Sent: Wednesday, March 7, 2018 7:09 PM
To: ***@trafficserver.apache.org<mailto:***@trafficserver.apache.org>
Subject: HLS streaming issue

Dear All,

We are working on HLS streaming. Once # of concurrent sessions reach around 4K, we are experiencing issues regarding the cache algorithm.

Although Cache.config is set to 10minutes for ts files ( url_regex=(.*)\.(ts)$ ttl-in-cache=10m ), lots of requests will be directed to origin server which previously get http 200 from origin server, ATS continues to direct the request to origin server.

Have anyone experienced such an issue? Or does anyone have an idea how to solve this issue?

Please see origin server log belowâŠ ATS sending lots of requests for the same file at the same time although it gets http 200.

176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 181000 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 78192 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"

Please see ATS log at the same time;

1520425086.244 chi=52.214.59.217 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=39879 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.254 chi=34.245.110.191 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40006 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.257 chi=34.242.200.72 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40043 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.258 chi=34.253.240.24 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=39944 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.259 chi=54.171.6.29 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=39951 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.259 chi=34.245.98.243 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40047 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.411 chi=34.245.117.91 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40020 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.414 chi=34.253.239.40 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40150 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.421 chi=34.243.255.27 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40176 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.427 chi=34.249.94.171 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40083 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.428 chi=52.48.159.168 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40028 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.431 chi=54.246.245.203 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=38984 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.435 chi=34.249.44.167 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40103 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.435 chi=54.171.125.63 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=37816 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.435 chi=54.171.125.63 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40198 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"

Thanks & kind regards,
Burak

[http://www.turkcell.com.tr/downloads/bireysel/img/Tcelldis.gif]<http://turkcell.li/iyaani>

Bu elektronik posta ve onunla iletilen butun dosyalar sadece gondericisi tarafindan almasi amaclanan yetkili gercek ya da tuzel kisinin kullanimi icindir. Eger soz konusu yetkili alici degilseniz bu elektronik postanin icerigini aciklamaniz, kopyalamaniz, yonlendirmeniz ve kullanmaniz kesinlikle yasaktir ve bu elektronik postayi derhal silmeniz gerekmektedir.

TURKCELL bu mesajin icerdigi bilgilerin doÄruluÄu veya eksiksiz oldugu konusunda herhangi bir garanti vermemektedir. Bu nedenle bu bilgilerin ne sekilde olursa olsun iceriginden, iletilmesinden, alinmasindan ve saklanmasindan sorumlu degildir. Bu mesajdaki gorusler yalnizca gonderen kisiye aittir ve TURKCELLin goruslerini yansitmayabilir

Bu e-posta bilinen butun bilgisayar viruslerine karsi taranmistir.

This e-mail and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the intended recipient you are hereby notified that any dissemination, forwarding, copying or use of any of the information is strictly prohibited, and the e-mail should immediately be deleted.

TURKCELL makes no warranty as to the accuracy or completeness of any information contained in this message and hereby excludes any liability of any kind for the information contained therein or for the information transmission, reception, storage or use of such in any way whatsoever. The opinions expressed in this message belong to sender alone and may not necessarily reflect the opinions of TURKCELL.

This e-mail has been scanned for all known computer viruses.

Vasanth Mathivanan

2018-03-08 09:59:30 UTC

Permalink

Can you check this below Config ?

Thanks & Regards
Vasanth M
Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10

From: BURAK EGERCI<mailto:***@turkcell.com.tr>
Sent: Thursday, March 8, 2018 1:26 PM
To: ***@trafficserver.apache.org<mailto:***@trafficserver.apache.org>
Subject: RE: HLS streaming issue

Hi,

I am not aware of Hdr_rw_live.config, I believe this is custom config, please confirm.

Thanks & kind regards,
Burak

From: Vasanth Mathivanan [mailto:***@evolutiondigital.com]
Sent: Thursday, March 8, 2018 9:30 AM
To: ***@trafficserver.apache.org
Subject: COMMERCIAL:RE: HLS streaming issue

Hi,
Hdr_rw_live.config File ? We are Using Our Environment Using Like this ?

This Condition based on Every M3u8 file store 5 secs then automatically call m3u8 and ts file every 150 sec store all ts then remove after 151 sec .All are based on Our live setup like our setup creating M3u8 file every 4 secs so we entered in 5sec in condition and 60 files stored in 150 secs again delete and make new one .. can you send HLS fragments or chunk value ?

cond %{READ_RESPONSE_HDR_HOOK} [AND]
cond %{PATH} /\.m3u8/
cond %{STATUS} >199 [AND]
cond %{STATUS} <400
rm-header Pragma
rm-header Expires
set-header Cache-Control "max-age=5, public"
cond %{READ_RESPONSE_HDR_HOOK} [AND]
cond %{PATH} /\.ts/
cond %{STATUS} >199 [AND]
cond %{STATUS} <400
rm-header Pragma
rm-header Expires
set-header Cache-Control "max-age=150, public"

Thanks & Regards
Vasanth M

Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10

From: Jeremy Payne<mailto:***@gmail.com>
Sent: Thursday, March 8, 2018 7:21 AM
To: ***@trafficserver.apache.org<mailto:***@trafficserver.apache.org>
Subject: Re: HLS streaming issue

I've been running daily load tests for almost the last 6+ months.
6000 simulated users, which is 12k connections at around 17-18gbps.
Before I started playing with the collapse_forwarding plugin, I would see
requests 'leaking' to the origin, when those requests should have been served by cache.
Now that I have collapse_forwarding tuned to my testing environment, combined with the associated
records.config parameters, I no longer see 'leaking' origin requests. This goes for both
100% cached HLS VOD(v-segs=2.7MB a-segs=66KB) and for simulated HLS live.

On Wed, Mar 7, 2018 at 8:58 AM, BURAK EGERCI <***@turkcell.com.tr<mailto:***@turkcell.com.tr>> wrote:
Hi,
Thanks for the quick response.

I m using m3u8 file. But not the main m3u8 as I want to specifiy the HLS profile. Lets assume index.m3u8 includes 01.m3u8 and 02.m3u8, I m requesting 01.m3u8.

Once ~4000 concurrent clients are accessing, the issue can be visible. It is not visible if the accessing client number is less than 4000.

Currently Ram storage is configured.

Thanks for Collapse Forwarding algorithm, I will set this up, that may solve the issue. Will let you know after trial.
Here is the records.config;

CONFIG proxy.config.accept_threads INT 5
CONFIG proxy.config.admin.admin_user STRING admin
CONFIG proxy.config.admin.autoconf_port INT 8083
CONFIG proxy.config.admin.number_config_bak INT 3
CONFIG proxy.config.admin.user_id STRING ats
CONFIG proxy.config.alarm.abs_path STRING NULL
CONFIG proxy.config.alarm.bin STRING example_alarm_bin.sh
CONFIG proxy.config.alarm_email STRING ats
CONFIG proxy.config.allocator.debug_filter INT 0
CONFIG proxy.config.allocator.dontdump_iobuffers INT 0
CONFIG proxy.config.allocator.enable_reclaim INT 0
CONFIG proxy.config.allocator.hugepages INT 0
CONFIG proxy.config.allocator.max_overage INT 3
CONFIG proxy.config.allocator.thread_freelist_size INT 1024
CONFIG proxy.config.body_factory.enable_customizations INT 1
CONFIG proxy.config.body_factory.enable_logging INT 0
CONFIG proxy.config.body_factory.response_suppression_mode INT 0
CONFIG proxy.config.body_factory.template_sets_dir STRING etc/trafficserver/body_factory
CONFIG proxy.config.cache.control.filename STRING cache.config
CONFIG proxy.config.cache.enable_read_while_writer INT 1
CONFIG proxy.config.cache.hostdb.sync_frequency INT 0
CONFIG proxy.config.cache.hosting_filename STRING hosting.config
CONFIG proxy.config.cache.http.compatibility.4-2-0-fixup INT 0
CONFIG proxy.config.cache.limits.http.max_alts INT 5
CONFIG proxy.config.cache.max_doc_size INT 0
CONFIG proxy.config.cache.min_average_object_size INT 131072
CONFIG proxy.config.cache.mutex_retry_delay INT 2
CONFIG proxy.config.cache.permit.pinning INT 0
CONFIG proxy.config.cache.ram_cache.algorithm INT 1
CONFIG proxy.config.cache.ram_cache.compress INT 0
CONFIG proxy.config.cache.ram_cache.size INT 34359738368
CONFIG proxy.config.cache.ram_cache.use_seen_filter INT 1
CONFIG proxy.config.cache.ram_cache_cutoff INT 1073741824
CONFIG proxy.config.cache.target_fragment_size INT 1048576
CONFIG proxy.config.cache.threads_per_disk INT 8
CONFIG proxy.config.cluster.cluster_configuration STRING cluster.config
CONFIG proxy.config.cluster.cluster_port INT 8086
CONFIG proxy.config.cluster.ethernet_interface STRING lo
CONFIG proxy.config.cluster.log_bogus_mc_msgs INT 1
CONFIG proxy.config.cluster.mc_group_addr STRING 224.0.1.37
CONFIG proxy.config.cluster.mc_ttl INT 1
CONFIG proxy.config.cluster.mcport INT 8089
CONFIG proxy.config.cluster.rsport INT 8088
CONFIG proxy.config.config_dir STRING etc/trafficserver
CONFIG proxy.config.core_limit INT -1
CONFIG proxy.config.diags.debug.enabled INT 0
CONFIG proxy.config.diags.debug.tags STRING http.*|dns.*
CONFIG proxy.config.diags.show_location INT 0
CONFIG proxy.config.dns.max_dns_in_flight INT 2048
CONFIG proxy.config.dns.nameservers STRING NULL
CONFIG proxy.config.dns.resolv_conf STRING /etc/resolv.conf
CONFIG proxy.config.dns.round_robin_nameservers INT 0
CONFIG proxy.config.dns.search_default_domains INT 0
CONFIG proxy.config.dns.splitDNS.enabled INT 0
CONFIG proxy.config.dns.url_expansions STRING NULL
CONFIG proxy.config.dns.validate_query_name INT 0
CONFIG proxy.config.dump_mem_info_frequency INT 0
CONFIG proxy.config.env_prep STRING example_prep.sh
Thanks & kind regards,
Burak

From: Vasanth Mathivanan [mailto:***@evolutiondigital.com<mailto:***@evolutiondigital.com>]
Sent: Wednesday, March 7, 2018 5:09 PM
To: ***@trafficserver.apache.org<mailto:***@trafficserver.apache.org>
Subject: COMMERCIAL:RE: HLS streaming issue

Hi,
I have Lot of Questions In front of You:

1.Are You using M3u8 or Direct ts file ?
2.How Many Clients Accessing ( because One client to hits Directly from origin the Other Clients will be loaded on Cache ?
3.Which Storage Method Using like Ram or Hard Disk ?
4.Which Algorithm Using in hdr_rw_config file?
5.Some times happen multiple Concurrent requests so you do need to add Collapse Forwarding Algorithm
6.What Parameter using in records. Config file ?

Thanks & regards

Vasanth M

Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10

From: BURAK EGERCI<mailto:***@turkcell.com.tr>
Sent: Wednesday, March 7, 2018 7:09 PM
To: ***@trafficserver.apache.org<mailto:***@trafficserver.apache.org>
Subject: HLS streaming issue

Dear All,

We are working on HLS streaming. Once # of concurrent sessions reach around 4K, we are experiencing issues regarding the cache algorithm.

Although Cache.config is set to 10minutes for ts files ( url_regex=(.*)\.(ts)$ ttl-in-cache=10m ), lots of requests will be directed to origin server which previously get http 200 from origin server, ATS continues to direct the request to origin server.

Have anyone experienced such an issue? Or does anyone have an idea how to solve this issue?

Please see origin server log below ATS sending lots of requests for the same file at the same time although it gets http 200.

176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 181000 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 78192 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"
176.235.11.9 - - [07/Mar/2018:15:18:04 +0300] "GET /Fashionbox/20180125T023356-06-357677.ts HTTP/1.1" 200 665896 "-" "User-Agent"

Please see ATS log at the same time;

1520425086.244 chi=52.214.59.217 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=39879 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.254 chi=34.245.110.191 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40006 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.257 chi=34.242.200.72 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40043 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.258 chi=34.253.240.24 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=39944 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.259 chi=54.171.6.29 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=39951 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.259 chi=34.245.98.243 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40047 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.411 chi=34.245.117.91 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40020 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.414 chi=34.253.239.40 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40150 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.421 chi=34.243.255.27 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40176 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.427 chi=34.249.94.171 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40083 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.428 chi=52.48.159.168 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40028 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.431 chi=54.246.245.203 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=38984 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.435 chi=34.249.44.167 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40103 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.435 chi=54.171.125.63 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=37816 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"
1520425086.435 chi=54.171.125.63 phn=tvgbzcdnes02 php=80 shn=176.235.11.2 url=http://tvgbztcdnes02.channels.turkcelltvplus.com.tr/Fashionbox/20180125T023356-06-357677.ts cqhm=GET cqhv=HTTP/1.1 pssc=200 ttms=40198 b=665896 sssc=200 sscl=665896 cfsc=FIN pfsc=FIN crc=TCP_MISS phr=DIRECT pqsn=176.235.11.2 uas="User-Agent"

Thanks & kind regards,
Burak

[http://www.turkcell.com.tr/downloads/bireysel/img/Tcelldis.gif]<http://turkcell.li/iyaani>

Bu elektronik posta ve onunla iletilen butun dosyalar sadece gondericisi tarafindan almasi amaclanan yetkili gercek ya da tuzel kisinin kullanimi icindir. Eger soz konusu yetkili alici degilseniz bu elektronik postanin icerigini aciklamaniz, kopyalamaniz, yonlendirmeniz ve kullanmaniz kesinlikle yasaktir ve bu elektronik postayi derhal silmeniz gerekmektedir.

TURKCELL bu mesajin icerdigi bilgilerin doðruluðu veya eksiksiz oldugu konusunda herhangi bir garanti vermemektedir. Bu nedenle bu bilgilerin ne sekilde olursa olsun iceriginden, iletilmesinden, alinmasindan ve saklanmasindan sorumlu degildir. Bu mesajdaki gorusler yalnizca gonderen kisiye aittir ve TURKCELLin goruslerini yansitmayabilir

Bu e-posta bilinen butun bilgisayar viruslerine karsi taranmistir.

This e-mail and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you are not the intended recipient you are hereby notified that any dissemination, forwarding, copying or use of any of the information is strictly prohibited, and the e-mail should immediately be deleted.

TURKCELL makes no warranty as to the accuracy or completeness of any information contained in this message and hereby excludes any liability of any kind for the information contained therein or for the information transmission, reception, storage or use of such in any way whatsoever. The opinions expressed in this message belong to sender alone and may not necessarily reflect the opinions of TURKCELL.

This e-mail has been scanned for all known computer viruses.