Removing duplicate lines from files keeping the original order

Removing duplicate lines from files keeping the original order

To remove the duplicate lines preserving their order in the file use:

The script keeps an associative array with indices equal to the unique lines of the file and values equal to their occurrences. For each line of the file, if the line occurrences are zero then it increases them by one and prints the line, otherwise it just increases the occurrences without printing the line. Piping a bunch of commands we can overcome this issue:

Suppose we have the following file:

prepends the order number in each line.

Source: iridakos.com

“This is a solar-powered website, which means it sometimes goes offline”

“This is a solar-powered website, which means it sometimes goes offline”

This website runs on a solar powered server located in Barcelona, and will go off-line during longer periods of cloudy weather. However, this limied energy storage became problematic as the days got shorter. This set-up would keep the website online during one or two days of bad weather, after which it will go off-line.

Source: solar.lowtechmagazine.com

Building Facebook’s Service Encryption Infastructure

Building Facebook’s Service Encryption Infastructure

For example, if engineers have access to a host running services with a particular identity, they should not be able to decrypt traffic from hosts from other identities — or even other hosts from the same identity forever. In addition to building Kerberos support and in-memory ticket caching into Thrift, to pre-fetch tickets, we also tuned thread pools to scale to large services and clients with tens of thousands of concurrent connections and made multiple improvements to the MIT Kerberos library. The ICA then verifies the identity certificate and generates three STEKs for the service using a key derivation function (KDF), like so:

The identity comes directly from the client identity in the mutually authenticated TLS connection.

Source: code.fb.com

Free GDPR and PCI DSS compliance test – all in one

Free GDPR and PCI DSS compliance test – all in one

The non-intrusive online test quickly verifies relevant GDPR and PCI DSS requirements, checks CMS security and runs a privacy check. ImmuniWeb, a global provider of web, mobile and API security testing and risk ratings, has now added a GDPR compliance check to its website security test, one of the most popular free tests provided within ImmuniWeb’s community offering. To test how largest European websites adhere to GDPR requirements related to web applications, we selected the 100 most visited websites (excluding local versions of global brands e.g. Google or Facebook) of the 28 European member states and ran the following non-intrusive checks:

Below are the results per county:

Ilia Kolochenko, CEO and Founder of ImmuniWeb, comments: “ We can see laudable efforts aimed to improve web application security and adhere to GDPR requirements amid European companies.

Source: www.immuniweb.com

‘We’re Not Being Paranoid’: U.S. Warns of Spy Dangers of Chinese-Made Drones

‘We’re Not Being Paranoid’: U.S. Warns of Spy Dangers of Chinese-Made Drones

But the Department of Homeland Security is warning that drones manufactured by Chinese companies could pose security risks, including that the data they gather could be stolen. Lanier Watkins, a cyber-research scientist at Johns Hopkins University’s Information Security Institute, said his team discovered vulnerabilities in DJI’s drones. DJI is leading the industry on this topic and our technology platform has enabled businesses and government agencies to establish best practices for managing their drone data.

Source: www.npr.org

Docker Bug Allows Root Access to Host Filesystem

Docker Bug Allows Root Access to Host Filesystem

This is a variant of the time of check to time of use (TOCTOU) problem, specifically with the “docker cp” command, which copies files to and from containers. After the full path has been resolved, the resolved path is passed around a bit and then operated on a bit later (in the case of ‘docker cp’ it is opened when creating the archive that is streamed to the client),” Sarai said in his advisory on the problem. “The most likely case for this particular vector would be a managed cloud which let you (for instance) copy configuration files into a running container or read files from within the container (through “docker cp”),,” Sarai said via email.

Source: decipher.sc

Microsoft hints at new modern Windows OS with ‘invisible’ background updates

Microsoft hints at new modern Windows OS with ‘invisible’ background updates

We’ve seen various promises about Windows Updates being improved over the years, but Microsoft is now promising that “modern OS updates are invisibly done in the background; the update experience is deterministic, reliable, and instant with no interruptions!” Some of these hints at a modern OS could already apply to Windows as it exists today, but Microsoft also describes its focus on “enabling cloud-connected experiences that use the compute power of the cloud to enhance users experiences on their devices” for this modern OS. A modern OS has the right sensor support and posture awareness to enable the breadth of innovative form factors and applications that our partner ecosystem will deliver

Microsoft was rumored to be discussing plans for dual-screen devices and Windows Lite at the company’s Build conference earlier this month.

Source: www.theverge.com

What’s the Magic Behind Graphene’s ‘Magic’ Angle?

What’s the Magic Behind Graphene’s ‘Magic’ Angle?

For that angle, the models differ only slightly, because eliminating tunneling in the fully aligned regions doesn’t have a big impact: The TKV version of the story predicts 1.09 degrees for the magic angle versus 1.05 in MacDonald’s. MacDonald and Vishwanath peg the next-largest magic angle at 0.5 and less than 0.3 degrees, respectively; as experimenters refine their techniques, they’ll be able to pit the models against each other. What’s more, Vishwanath added, the stresses and distortions introduced by experimenters’ efforts to clamp graphene sheets together at particular angles tend to reduce the half-aligned area, a factor that the TKV model ignores.

Source: www.quantamagazine.org

Choosing Ruby on Rails for web development project in 2019

Choosing Ruby on Rails for web development project in 2019

[source]

Without Rails, Ruby might not have become a popular programming language for web applications. [source]

There are easy ways to integrate Rails with popular front-end frameworks, as well as other third-party technologies for web development. [source]

Contained in Ruby on Rails is a whole suite of solutions for various common problems that developers face when building applications.

Source: www.ideamotive.co

Angular v8.0

Angular v8.0

The recommended method to transition is to use relative paths within the source stylesheet. So, for example, the release becomes the version for experimental packages. This allows us to better manage our experimental packages (they are always monotonic with time), and it allows users to know if they are not matching versions between their packages in their projects, without relying on looking up the npm registry or other sources.

Source: github.com